MATLAB Answers


Manual Calculation of Entropy not matching matlab

Asked by thatguy14 on 7 Apr 2019
Latest activity Commented on by Walter Roberson
on 7 Apr 2019
Hello, I am having trouble calculating entropy "by hand." I was doing this to learn more about it but now side-tracked because what I am doing doesn't result in the same answer as matlab. Here is my code:
matrix = abs(randn(1,256));
nbins = 256;
min_mat = min(matrix);
max_mat = max(matrix);
nbins_step = (max_mat-min_mat)/256;
bins = min_mat:nbins_step:max_mat;
bins(1) = bins(1)-nbins_step; %Adjust so that we can always use "greater than" bottom bin edge
%Count the number of occurances per bin
for ii = 1:nbins
pdist_bins(ii) = sum(matrix > bins(ii) & matrix <= bins(ii+1));
sum(pdist_bins) %Make sure that we have the same number of elements as we started with
pdist_norm = pdist_bins./numel(matrix); %Normalize
sum(pdist_norm) %Make sure adds to 1
%Remove 0s
pdist_no_zero = pdist_norm;
pdist_no_zero(pdist_no_zero == 0) = [];
entropy_calc = -sum(pdist_no_zero.*log2(pdist_no_zero))
%Matlab version
t = entropy(matrix)
I used my code on a simple example for a matrix [1 1 1 1 1 1 2 2 2 3 3 3] and got the correct answer (whereas matlab outputs 0...)
I am missing something but not sure. I think it is related to my choice of bins which may differ to matlabs imhist but I didn't think the result would differ by a lot (which I think it does).
Any help would be appreciated, thanks!


Sign in to comment.


1 Answer

Answer by Walter Roberson
on 7 Apr 2019
 Accepted Answer

To understand this you need to refer to the Tips section, where it says that anything other than logical is converted to uint8. Internally this is done by im2uint8(). im2uint8() assumes that floating point values are in the range 0 to 1, so when you pass in floating point values greater than 1, it assumes they should all saturate to the maximum uint8 value.


oh! Okay that's simple and also explains why the entropy returned by matlab is less than a hand calculation. So I guess I am doing it correctly! Thanks
The context is important for this purpose: entropy() is an Image Processing Toolbox function, and floating point data for images is expected to be in the range 0 to 1.

Sign in to comment.