MATLAB Answers

Manual Calculation of Entropy not matching matlab

7 views (last 30 days)
Hello, I am having trouble calculating entropy "by hand." I was doing this to learn more about it but now side-tracked because what I am doing doesn't result in the same answer as matlab. Here is my code:
matrix = abs(randn(1,256));
nbins = 256;
min_mat = min(matrix);
max_mat = max(matrix);
nbins_step = (max_mat-min_mat)/256;
bins = min_mat:nbins_step:max_mat;
bins(1) = bins(1)-nbins_step; %Adjust so that we can always use "greater than" bottom bin edge
%Count the number of occurances per bin
for ii = 1:nbins
pdist_bins(ii) = sum(matrix > bins(ii) & matrix <= bins(ii+1));
sum(pdist_bins) %Make sure that we have the same number of elements as we started with
pdist_norm = pdist_bins./numel(matrix); %Normalize
sum(pdist_norm) %Make sure adds to 1
%Remove 0s
pdist_no_zero = pdist_norm;
pdist_no_zero(pdist_no_zero == 0) = [];
entropy_calc = -sum(pdist_no_zero.*log2(pdist_no_zero))
%Matlab version
t = entropy(matrix)
I used my code on a simple example for a matrix [1 1 1 1 1 1 2 2 2 3 3 3] and got the correct answer (whereas matlab outputs 0...)
I am missing something but not sure. I think it is related to my choice of bins which may differ to matlabs imhist but I didn't think the result would differ by a lot (which I think it does).
Any help would be appreciated, thanks!


Sign in to comment.

Accepted Answer

Walter Roberson
Walter Roberson on 7 Apr 2019
To understand this you need to refer to the Tips section, where it says that anything other than logical is converted to uint8. Internally this is done by im2uint8(). im2uint8() assumes that floating point values are in the range 0 to 1, so when you pass in floating point values greater than 1, it assumes they should all saturate to the maximum uint8 value.


thatguy14 on 7 Apr 2019
oh! Okay that's simple and also explains why the entropy returned by matlab is less than a hand calculation. So I guess I am doing it correctly! Thanks
Walter Roberson
Walter Roberson on 7 Apr 2019
The context is important for this purpose: entropy() is an Image Processing Toolbox function, and floating point data for images is expected to be in the range 0 to 1.

Sign in to comment.

More Answers (0)

Sign in to answer this question.