Matlab numpy array: AttributeError: 'array.array' object has no attribute 'fromstring'

I'm having some issues working with numpy in Matlab since moving to updated versions of Matlab, python and numpy.

>> version
ans =
    '9.9.0.1467703 (R2020b)'
>> pyenv
ans = 
  PythonEnvironment with properties:

          Version: "3.9"
       Executable: "C:\Users\ben.george\AppData\Local\Programs\Python\Python39\python.EXE"
          Library: "C:\Users\ben.george\AppData\Local\Programs\Python\Python39\python39.dll"
             Home: "C:\Users\ben.george\AppData\Local\Programs\Python\Python39"
           Status: Loaded
    ExecutionMode: InProcess
        ProcessID: "7364"
      ProcessName: "MATLAB"
      

In this python environment I have numpy 1.19.4 installed.

The code below has previous worked to create a numpy array, but now gives an error.

>> p = 1; % This works just fine
>> p = py.numpy.array(p)
p = 

  Python ndarray with properties:

           T: [1×1 py.numpy.ndarray]
        base: [1×1 py.NoneType]
      ctypes: [1×1 py.numpy.core._internal._ctypes]
        data: [1×1 py.memoryview]
       dtype: [1×1 py.numpy.dtype]
       flags: [1×1 py.numpy.flagsobj]
        flat: [1×1 py.numpy.flatiter]
        imag: [1×1 py.numpy.ndarray]
    itemsize: [1×1 py.int]
      nbytes: [1×1 py.int]
        ndim: [1×1 py.int]
        real: [1×1 py.numpy.ndarray]
       shape: [1×0 py.tuple]
        size: [1×1 py.int]
     strides: [1×0 py.tuple]

    1.0

>> p = [1, 2, 3]; % This will fail
>> p = py.numpy.array(p)
Python Error: AttributeError: 'array.array' object has no attribute 'fromstring' 

For reasons which I cannot entirely remember, the whole block that this comes from is as follows, but now gets stuck creating the numpy array (see above).

if size(p,1) == 1
    p = py.numpy.array(p);
else
    p = double(p);
    sz = uint16(size(p));
    p = reshape(p,[1 numel(p)]);    % Conversion to Python is only supported for 1-N vectors.
    p = py.numpy.array(p);          % if empty, type is always set to double: https://github.com/numpy/numpy/issues/6028
    p = p.reshape(num2cell(fliplr(sz)));
    t = uint16(0:length(sz)-1);
    t(end-[1 0]) = t(end-[0 1]);
    p = p.transpose(num2cell(t));
end

This code worked in a previous combination of Matlab, python and numpy around two years ago. Sorry, but I don't recall the exact versions used at that time.

Answer 1

[Martin Caron]

What I find most troubling in this error is that matlab and python apparently communicate through strings. Why use strings in the background when those are definitely double arrays in the background. Anyhow, here is an example on how to extract from raw bytes. There seems to be a 72 bytes header (I did not investigate what it actually contains).

% Create some array
a = round(10*rand(2,3,4));

% Grab raw bytes
b = getByteStreamFromArray(a);

% Grab its shape
msize = size(a);

% Hardcoded header size found empirically (maybe should find some doc to
% justify this)
header = 72;

% Create numpy array from raw bytes buffer
d = py.numpy.frombuffer(b(header+1:end)).reshape(int32(msize));

Funny enough, it seems that you can then directly convert back to matlab using, although the values seem to be in a different order than in the original value "a".

e = double(d);

Answer 2

[Andy]

Martin's answer seemed promising, but I fear it exhibits the same issue in terms of how MATLAB is passing data to Python under the hood. getByteStreamFromArray() returns a row vector, which fails to make its way to Python with the same AttributeError. Has anyone actually gotten this to work on Python >= 3.9?

I was actually able to create a Numpy array in Python 3.9 from within MATLAB using Numpy's fromstring() method. Formulated for MATLAB, it looks like this:

>> x = py.numpy.fromstring('1 2 3 4 5', py.numpy.float64, int8(-1), char(' '));

It seems possible but is certainly less than ideal to pass data to Numpy by doing something like this:

a = rand(5, 5);
af = a';  % convert from F to C ordering
n = py.numpy.fromstring(num2str(af(:)'), py.numpy.float64, int8(-1), char(' ')).reshape(int32(size(a)));

I'm concerned about loss of precision here, and will need to do some extra work to handle arrays larger than 2D and to ensure the resulting ndarray dtype is correct, but this approach at least seems to work.

As a side note, I was unable to find any documentation about the size of the byte stream header in the getByteStreamFromArray() response, but noticed that it appeared to depend on the number of dimensions the array has. After a little fiddling, I believe the header is sized according to:

header = 64 + (ceil(ndims(a)/2) - 1) * 8;

Answer 3

[Markus Koschi]

I had the same problem and solved it by downgrading Python to version 3.6, since array.array: tostring() and fromstring() methods have been removed in Python 3.9.

Now, I get (using R2020b Update 1):

>> p = [1, 2, 3]; % This will not fail anymore
>> p = py.numpy.array(p)

p = 

  Python ndarray:

     1     2     3

    Use details function to view the properties of the Python object.

    Use double function to convert to a MATLAB array.

[Chris]

Hi Markus,

are you aware of any way to make this work without downgrading Python?

Answer 4

[Meme Young]

You know, Python always has this or that little glitches that happen when the version changes... so I really hate it

Answer 5

[Al Danial]

Regarding @Ben 's initial issue, the matlab/python version compatibility table, https://www.mathworks.com/content/dam/mathworks/mathworks-dot-com/support/sysreq/files/python-compatibility.pdf, shows that v2020b does not support Python 3.9 so that's the likely cause of the array conversion issue.

It's highly doubtful that NumPy array conversion--in either direction--is done through strings because such conversions happen too quickly.

If you want a general-purpose Python-to-MATLAB and MATLAB-to-Python variable converters, try out https://github.com/Apress/python-for-matlab-development/code/matlab_py/mat2py.m and https://github.com/Apress/python-for-matlab-development/code/matlab_py/py2mat.m