Error Exporting Table Using Parquetwrite (multidimensional array)

I have a 2x200 table I am trying to export it as a parquet file and I have been recieving the following error after calling:

parquetwrite('test.parq',gTruth)

Error using parquetwrite

T.Apple{1} is a multidimensional array. Nontabular cell array elements must be empty or 1-dimensional.

I realize that this is due to the formatting of the 2nd column. I need to keep the 4x1 format of the values in the 2nd cell as they represent bounding boxes. Each cell holds the locations of objects corisponding to the image located at the associated file path. How could I restructure my table to export this data? I attached the table in the file below.

Thank you ahead of time for suggestions!

Answer 1

[dpb]

parquetwrite

load gTruth
for i=1:height(gTruth), gTruth.Apple(i)={gTruth.Apple{i}(:)};end
parquetwrite('test.parq',gTruth)
t2 = parquetread("test.parq");
head(t2)

                          imageFilename                              Apple    
    _________________________________________________________    _____________

    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0065 - Copy.JPG"    {32×1 double}
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0066 - Copy.JPG"    {32×1 double}
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0067 - Copy.JPG"    {48×1 double}
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0068 - Copy.JPG"    {24×1 double}
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0070 - Copy.JPG"    { 8×1 double}
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0071 - Copy.JPG"    { 8×1 double}
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0072 - Copy.JPG"    {20×1 double}
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0073 - Copy.JPG"    { 4×1 double}

for i=1:height(t2), t2.Apple(i)={reshape(t2.Apple{i},[],4)};end
head(t2)

                          imageFilename                                 Apple       
    _________________________________________________________    ___________________

    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0065 - Copy.JPG"    { 8×4 double      }
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0066 - Copy.JPG"    { 8×4 double      }
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0067 - Copy.JPG"    {12×4 double      }
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0068 - Copy.JPG"    { 6×4 double      }
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0070 - Copy.JPG"    { 2×4 double      }
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0071 - Copy.JPG"    { 2×4 double      }
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0072 - Copy.JPG"    { 5×4 double      }
    "C:\Users\etbuc\OrBot\AppleAITestImg\IMG_0073 - Copy.JPG"    {[259 132 215 222]}

I'd never heard of a parquet file before...looks like you would have to reshape the 2D arrays to 1D to write with MATLAB toolset; whether there's a way for a parguet file to contain a 2D array if one had more capable toolset I dunno. But, while somewhat inconvenient, it's not too difficult to rearrange to write; of course you'll have to rearrange back on reading...

[Ethan Buchanan]

Thank you,

I know its a bit random, I couldn't find a better format to export the data. This will work well

[Walter Roberson]

I see that the file format itself permits nesting -- but whether the MATLAB interface supports that is a different matter.