How can I convert a dataset of numbers into a dataset of logarithms of these numbers

Hi

I have imported an Excel dataset of real-time GDP. Which is a dataset of numbers in a triangular format.

So I want to convert this dataset into logs of these numbers. Is there any fast way??

Many thanks

[Star Strider]

What is a ‘triangular format’?

[John D'Errico]

By using the log function?

[Panty]

The problem is that I am totally newly with matlab. How can I use the log function for the whole dataset?

Answer 1

[Star Strider]

You need to load the dataset into the MATLAB workspace (use xlsread), then simply use the logarithm function of your choice on the entire dataset. Your options are: log (Napierian logarithms), log2 (base 2 logarithms), or log10 (base 10 logarithms). Then save to the file type of your choice. (I suggest a ‘.mat’ file,)

For example:

DataSet = randi(10, 5, 3)
LogDataSet = log10(DataSet)

[Star Strider]

I have no idea what you are doing. I can’t find any documentation on the ‘dataset’ data type, so you must have created it somehow. If your dataset exists in your workspace as a double array, you should have no problem taking log10 of it.

[Panty]

Is that maybe because I got some negative values or numbers like 2039e+12 in my dataset?

[Star Strider]

Negative values create complex logarithms, but it's best to simply take the logs of the positive values.

There is a datasetfun function I haven’t used (it operates similarly to cellfun), but I doubt it would work easily in your situation.

Something similar to this would probably work in your situation:

Data = randi(10, 5, 3)-2
DataPos = find(Data > 0)
LData = log10(Data(DataPos))
LogData = Data;
LogData(DataPos) = LData

The code works. The problem is taking the antilog of it if you want to do that, because negative or zero values are valid logarithms.

Entries such as 2039e+12 aren’t problems, since log10(2039e+12) = 15.3094e+000. This is well within MATLAB’s floating-point ability.

[Panty]

Thank you very much for your help Star Strider. I will try to fix it later and I will let you know. Much appreciated.

[Star Strider]

My pleasure!

You can certainly use datasetfun to take the log10 of your data, but selecting for only positive values might require nested datasetfun calls, likely less efficient than converting your data to double and doing the sort of thing I did in my code. You might want to use the table functions instead, since the documentation for the dataset data type says:

• The dataset data type might be removed in a future release. To work with heterogeneous data, use the MATLAB® table data type instead. See MATLAB table documentation for more information.

[Panty]

Goodmorning Star Strider,

I can log transform my data only when they appear as a 'double array'. When I import them as 'dataset' I cant do it.

I attach you the workspace dataset I am using. Please check if you can log this one.! Many thanks

[Star Strider]

This seems to work:

D = load('Real GDP_dataset_workspace.mat')
DSC = struct2cell(D);
DS = DSC{1};
Data = double(DS);

DataPos = find(Data > 0);
LData = log10(Data(DataPos));
LogData = Data;
LogData(DataPos) = LData;       % Desired Result (Output)

It’s too large for me to look through the entire array, but it works as intended for the smaller test data, so I assume it is working on your full array as well.

Note that the DS variable uses curly braces ‘{}’ to address the contents of the cell as the dataset array.

[Panty]

It looks really clever :) So it seems that you have changed the structure from 'dataset' into 'double array'. And it does work. But I can get the same results by simply using:

data=xlsread('.................'); Logdata=log10(data);

and I got the same exactly results as yours. But both mine and yours log results are in a double array structure. But my goal is to keep the structure as a 'dataset'. So I can have the logGDP in a 'dataset'. NOT in a 'double array'. Any idea how???? I realy dont know how to thank you for helping me into this new world of Matlab.!

[Star Strider]

My pleasure!

It doesn’t seem to be possible to use datasetfun to do what you want, and to keep the data as a dataset. You need to take it out of the dataset just as I did. If you want to keep it as a dataset, either do as I did, and then create the result as a dataset, or do it in Excel. (I don’t have enough experience with Excel to help you with that.)

Note that your Logdata = log10(data) will not do what you specified. It will take the logs of the negative data as well, leaving you with complex numbers for them, and with -Inf for any that are zero.

[Panty]

I see that you are true about loging only the positive values using yours. Do you know then how we can create the results as a dataset? in matlab..not excel. Thank you!

[Star Strider]

I did everything I could to keep them as a dataset and couldn’t. The problem is that taking only the positive values creates vectors of unequal sizes in the dataset, and that is apparently not an option with them. I used the UniformOutput and DatasetOutput options, but without success. I also used my own inline function to select only positive values for the log, but for some reason (I believe it deleted the non-positive values), that failed. The datasetfun function only operates on dataset variables, not specific elements of the variables.

If you have R2013b or later, creating your data as a table using the table functions might allow you to do what you want and keep the string labels. I didn’t try that because you want to keep them as a dataset.

[Panty]

I see your point.. I will try to use the table function. It may well works for me. I will let you know.

Much appreciated !!

[Star Strider]

My pleasure!

The dataset class may be very useful in some situations, but it has serious limitations with respect to doing the sort of operation you want to do. The table class is also a core MATLAB set, and so likely portable amongst a number of different applications. I didn’t try table with your data, but I’ll give it a go if you have problems with it in your application.

Answer 2

[Panty]

I have already imported the dataset using 'Import Data' button and I have the dataset in my workspace with 'Value' dataset. And ROUTPUTQvQ19652014 is the name of my dataset.

I used: logReal_GDP= log10(ROUTPUTQvQ19652014)

BUT I GOT : Undefined function 'log10' for input arguments of type 'dataset'.

[Panty]

Is that maybe because I got some negative values or numbers like 2039e+12 in my dataset?