Using unique.m function on NaNs

Can someone please clarify if the result of unique([1 nan; 1 nan],'rows') should be

a) [1 nan; 1 nan] OR b) [1 nan]

Basically, in my version of matlab (2008b), I get a) as the answer and I think it's a bug as the answer should be b). Can someone please confirm if this is a feature or an issue. My mathworks support is limited because of $$ :(

Answer 1

[Sean de Wolski]

No. It's supposed to be a.

By definition nans are not equal to themselves so they're not equal in rows either.

isequal(nan,nan)

or

nan==nan

Answer 2

[Jan]

NaNs compared to anything replies FALSE, even NaN==NaN. This is defined by the IEEE754 conventions.

Therefore [1, NaN] compared to [1, NaN] is FALSE also. In consequence UNIQUE([[1, NaN; 1, NaN], 'rows') must reply two rows.

Answer 3

[Matt Fig]

In addition to what others have said, note this horribly named function:

A = [1 nan; 1 nan];
isequalwithequalnans(A(1,:),A(2,:))

[Matt Fig]

Where are:

ismemberwithequalnans

intersectwithequalnans

setdiffwithequalnans

etc.

[Sean de Wolski]

Some are on the FEX, though I think the authors took more care in naming them :-)

[Andrew Newell]

What's so bad about the name? It's long, but it says what it does.

[Sean de Wolski]

nanisequal does too and sticks in line with: nanmean, nanmedian, nancumsum etc.

[Matt Fig]

@Andrew, I think you would agree that there is more to a great function name that it being descriptive! How often do you name your functions, for instance:

performstatisticsonmydatathenplottheresults(A)

I just think it is a monstrosity (IMO) to have a 20 character function name, that's all. Why not, for example, use:

isequal(A,B,'nansequal') % Or similar

or a name like Sean de recommends?

Answer 4

[Andy]

If you have some application for which you need to ignore NaNs rather than take them into account, you could replace them with some other value which is easily identifiable as non-data. For example:

A; % your data
A(isnan(A)) = Inf; % replace NaN with Inf
A = unique(A,'rows'); % unique rows, where [1,Inf] == [1,Inf]
A(isinf(A)) = NaN; % convert back to NaN

It's a little hackish. I know there are various toolbox functions, like nanmean and nanstd, which calculate other statistics of a dataset while ignoring NaNs. But there is no nanunique or nansort. Also, the work in unique is done by calling sortrows which calls a MEX file, so you can't just edit it to take NaNs into account. So the above code seems like the cleanest quick solution.

[Sean de Wolski]

What if there are infs in there though? You don't want those incorrectly turned to nans or lost.

[Matt Fig]

I think the point is for the user to substitute a value which is not in the array. Although I have used automatic replacement software before, I usually use something like:

A(isnan(A)) = eps/(pi*exp(1))

[Rahul Chander]

I think i would need to precondition the data to deal with NaNs basically by either removing them or replacing them or interpolating. Thank you all.

[Andy]

@Sean de, Matt Fig is correct. I intended him to replace Inf with any "value which is easily identifiable as non-data". I was just using Inf as an example, since, for my purposes, Inf never comes up data. But Rahul should choose whatever flag value is appropriate for the application.