data.table equivalent in matlab

Hi everybody,

I am coming from R and used the data.table package a lot to filter, group, and aggregate large datasets. Is there an equivalent in Matlab?

Best,

[per isakson]

"large datasets" how large are these datasets compared to the physical memory of the computer?

[Simon493]

The datasets usually are between 12-14 gb, so they usually fit in the memory.

Answer 1

[Luna]

Hello,

Have you ever checked table and datastore? You can look at below links and their relevant properties and methods in documentation:

table

datastore

[Simon493]

Hi Luna,

Thank you for your answer. I know table (and functions like varfun) but I found them to be much slower than data.table from R. The data.table package in R works a bit like SQL, by using primary keys to quickly access large dataframes/tables. It also allows things like rolling joins etc. Is there an equivalent in matlab?

[Luna]

You should check datastore and databasedatastore I can recommend you. (if you are working with large tables)

I am sure you can find what functions you need with a little bit research on that. Also you can overview tall arrays, too.

What is DataStore?

datastore

databasedatastore

note: Primary keys are also exist in tables in Matlab, too.

Access Data in a Table

Join Tables (check innerjoin and outerjoin functions, too)

Add Delete Table Variables (check addvars | inner2outer | mergevars | movevars | removevars | rows2vars | splitvars | table)

Please don't forget to look at See Also section on the bottom of the pages. There are useful functions and topics in there.

note2: if you want to access a specific data in your table, use logical indexing instead of cellfun arrayfun or varfun. It will be way much faster. You should read Loren's article in here:

Logical Indexing

[Simon493]

Thank you very much! Your answer helps me a lot!

I will do research on the links you send me.

[Luna]

Your welcome :)