How to load parquet files where some columns are lists
17 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
I am trying to work with parquet files between python and MATLAB. Unfortutanely it seems matlab does not support reading parquet files if some of the columns are "unsupported types" such as lists/arrays/JSON (see https://ch.mathworks.com/help/releases/R2021a/matlab/import_export/datatype-mappings-matlab-parquet.html).
Is there maybe a way to force MATLAB to read in those columns as strings?
I thoughts about getting the parquet file header info with `parquetinfo` and force the VariableTypes of the columns with lists to be "string" but it seems the returned info file is read-only.
Thanks,
+Enea
1 Kommentar
Maik Berchten
am 17 Dez. 2021
We're faced a similar problem and ended up converting the array to a CSV string and storing that as UTF8 string while generating the parquet file. Thanks for reporting this (and we're looking forward to a solution that doesn't need a workaround)!
--maik
Antworten (1)
Divyanshu
am 26 Feb. 2024
Hi Enea,
The Parquet Files or Parquet Formatted data was not supported in the MATLAB version R2021a. However, in the later versions these file formats are supported as mentioned in the release notes of R2021b:
Also there are two functions 'parquetread' and 'parquetwrite' through which you can work with parquet files from MATLAB. Please refer the following documentation links to read more about these functions:
0 Kommentare
Siehe auch
Kategorien
Mehr zu JSON Format finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!