Filter löschen
Filter löschen

How to load parquet files where some columns are lists

15 Ansichten (letzte 30 Tage)
Enea Ceolini
Enea Ceolini am 13 Dez. 2021
Beantwortet: Divyanshu am 26 Feb. 2024
I am trying to work with parquet files between python and MATLAB. Unfortutanely it seems matlab does not support reading parquet files if some of the columns are "unsupported types" such as lists/arrays/JSON (see https://ch.mathworks.com/help/releases/R2021a/matlab/import_export/datatype-mappings-matlab-parquet.html).
Is there maybe a way to force MATLAB to read in those columns as strings?
I thoughts about getting the parquet file header info with `parquetinfo` and force the VariableTypes of the columns with lists to be "string" but it seems the returned info file is read-only.
Thanks,
+Enea
  1 Kommentar
Maik Berchten
Maik Berchten am 17 Dez. 2021
We're faced a similar problem and ended up converting the array to a CSV string and storing that as UTF8 string while generating the parquet file. Thanks for reporting this (and we're looking forward to a solution that doesn't need a workaround)!
--maik

Melden Sie sich an, um zu kommentieren.

Antworten (1)

Divyanshu
Divyanshu am 26 Feb. 2024
Hi Enea,
The Parquet Files or Parquet Formatted data was not supported in the MATLAB version R2021a. However, in the later versions these file formats are supported as mentioned in the release notes of R2021b:
Also there are two functions 'parquetread' and 'parquetwrite' through which you can work with parquet files from MATLAB. Please refer the following documentation links to read more about these functions:

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by