How can I store large strings in an array?

4 Ansichten (letzte 30 Tage)
Ed Frank
Ed Frank am 24 Mär. 2020
Kommentiert: John D'Errico am 24 Mär. 2020
Dear Matlab community,
I need to store large strings coming from fread in an array. With a conventional array, this fails with the "Out of memory." error.
I am not really into this topic but for handling large data sets I found two possible solutions which both do apparently not work for my situation:
1) Using matfile: This works great for arrays of numbers, but trying to use strings with a command like
matfileObj.stringArray(i) = char(fread(...));
results in the error "... is of type string. To use ... load the entire variable." However, I am not able to do this for the above-mentioned lack of memory.
2) Using tall: Unfortunately, the file that I work on is in a very ugly binary format that I think can't connect with a datastore. I could, however, initialise a string array of the needed length and generate a tall array from this. But now it seems I can't replace the initialisation strings with the real data; even the command
tallStringArray(i) = "";
fails with "For A(m,n,...) = B, m must be either a colon (:) or a tall logical vector." So apparently, I can't change single entries in the tall array.
Do you have any idea how I can store these large strings?
Many thanks for sharing your thoughts,
Ed
  3 Kommentare
Ed Frank
Ed Frank am 24 Mär. 2020
fread() from a binary file, converted to a string. This string has to be put in a string array, after this the next string is read from the file, put in the array and so on. The strings' length is known, but differs between the strings.
John D'Errico
John D'Errico am 24 Mär. 2020
You have large strings of varying sizes? Then learn about cell arrays, which can store them without wasting space because some are larger than others. If you use a regular array, then the array needs to be rectangular, so is far less effcieitn in terms of memory.

Melden Sie sich an, um zu kommentieren.

Antworten (0)

Kategorien

Mehr zu Large Files and Big Data finden Sie in Help Center und File Exchange

Produkte

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by