Why won't jsonencode encode my entire structure array?
18 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Andrew Mistele
am 4 Aug. 2020
Bearbeitet: Andrew Mistele
am 13 Aug. 2020
I have a 1x2 struct named "Vortices" that contains data outputs for two objects in a simulation, and is approximately 1.2 GB in size. Each struct in the struct array has 6 fields, each of which are 1x6000 struct arrays as well. I want to encode "Vortices" in JSON and write out to a "vortices.json" file, using the jsonencode() function, as below:
file_handle = fopen('vortices.json','w');
str_vortices = jsonencode(Vortices);
fprintf(file_handle,str_vortices);
"Vortices" structure:
However, I have tried this with structure outputs from several different simulations, and found that my file write-out never results in a JSON file larger than 2,097,152 KB. I have backtracked and found that this is because the jsonencode(Vortices) is not encoding the entire structure, as you can see the encoded string output from jsonencode() does not end with any JSON termination characters like "}" or "]":
I know that the structure I'm trying to encode is large, but I'm running MATLAB 64-bit (R2020a) and the sizes of these objects are well below what inputting "memory" indicates is my largest possible array size, 43.8 GB.
Since I haven't exceeded the maxium array size (or my allocated RAM), why would jsonencode() not be encoding the entire structure?
0 Kommentare
Akzeptierte Antwort
Gaurav Garg
am 13 Aug. 2020
Hey Andrew,
You can divide your data into small chunks and then enode those chunks (one-by-one) into a JSON file (using jsonencode).
You can start by opening a file in write mode, loading a chunk of data, writing the data into JSON, and repeating the same steps but opening the file in append mode this time.
It will lead to some discrepancies because the resultant JSON file will show the variable names multiple times.
For example -
I assume a struct 'data' with variable 'x' in it. For 2 data chunks, I would get something like -
{"x": values} {"x": values}
But, you can easily write a script to handle this case. HINT: Start off with small example to remove the second "x", "}{" and ":" symbols and then generalize the script for the whole JSON file (with all the chunks of data).
Using this approach, you can encode even larger data without any errors.
1 Kommentar
Weitere Antworten (0)
Siehe auch
Kategorien
Mehr zu JSON Format finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!