How would you store these data?

Hi all,

I'm hoping for some guidance and advice.

The data I work with are normally realtively large datasets with small number of rows and large number of columns..

They tipically consist in samples (rows) and features (columns).

However, the samples have specific relationships and can be grouped together in different ways (normally these groupings are defined by an acillary file we call metadata)

I will tipically carryout operations which will transform the data and at each step the new and old matrix need to be recorded (create a new variables).

In the past I have done this by using new variables, but it get quite confusing as the number of variables in the workspace increase quite dramatically.

In an attempt to improve this I started using structures, but these can be hard to index at times.

My latest attempt is to store these data in Tables, which works well, but again indexing starts getting tricky (we do alot of indexing in both directions (columns and rows)

).

So My hope is that someone who works with these type of data has found a good design/system to store and operate on these.

Some attached screenshots.

[Bob Thompson]

I think indexing anything more complex than a basic array is going to have it's own challanges.

The only other way to store data i know, in the layered style that you're doing is with cells. You won't get the labels the same way you do for the others, unless you want to add them as cell elements, but it does make calling nested elements a bit easier. Not sure if it's really what you're looking for, but it might be worth a shot. Keep in mind that you will need cellfun in order to perform the same action on multiple higher level cells.

[Goncalo Gouveia]

Thank you that is helpful, the labels become really important to make sure different elements dont get assigned to other samples. I haven't had the need to use celfun often, i know it can be really useful, would it also work on tables?

[Bob Thompson]

cellfun works on any element that's classified as a cell, so I don't know that it would work directly on a table, but will work on the contents.

You can look into some other commands, such as rowfun and varfun, but I have not personally used them before. Also, to be perfectly candid, all of these options are basically just for loops, so don't expect them to be all that fancy.

[Goncalo Gouveia]

Excellent! Thank you! I will check them out.