AP - even if I compile the code without the call to Reduce in your mexFunction, the program still crashes (compiled on R2014a, OS X 10.8.5). Assuming the code is placed in a file named, myConv.cpp, then
>> mex myConv.cpp
Building with 'Xcode Clang++'.
MEX completed successfully.
>> myConv(magic(4))
CRASH!
This crash, once the Reduce call has been commented out, points to a problem with the mxSetPr which uses the input parameter, prhs[0]. This is the same error experienced in a previous post of yours, so it isn't clear to me how - with this same code - you expect the code to work without crashing. Why not simplify your code in the mexFunction to the following
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
// create the appropriately sized output (?) matrix
mxArray *Bt1 = 0;
Bt1 = mxCreateNumericMatrix(
mxGetM(prhs[0]) >> 1, mxGetN(prhs[0]) >> 1, mxDOUBLE_CLASS, mxREAL);
// reduce the input matrix
Reduce(prhs[0], Bt1);
// destroy any created arrays
mxDestroyArray(Bt1);
}
Note how in the above we pass prhs[0] as the first input to Reduce, and that since we create a numeric matrix, then we must destroy it. Of course, the above will require some changes to your signature for Reduce. It met change from
void Reduce ( mxArray *B, mxArray *&Breduced );
to
void Reduce ( const mxArray *B, mxArray *Breduced );
Note how we need to add the const qualifier for the input matrix (which is prhs[0]), and that we are no longer passing the second input parameter by reference (?). Was that the intent with the Breduced, and if so, why?
And like the signature for Reduce was changed, we must do the same for convolve, changing it from
void convolve ( mxArray *Ain, mxArray *Kernel, mxArray *&Aout );
to
void convolve ( const mxArray *Ain, mxArray *Kernel, mxArray **Aout );
Note how we need to add the const qualifier for the input matrix (which is prhs[0]), and that the output matrix is mxArray **Aout, since we expect convolve to create this matrix for us. In the calling function, we just create a null pointer to an mxArray and want convolve to create it for us. This means that we must also change the manner in which we call convolve from
convolve( B, mask, BConvolved );
to
convolve( B, mask, &BConvolved );
where we pass the address of the pointer to BConvolved. The same will need to be done for the other call to convolve.
Due to this, the call to mexCallMATLAB will need to change slightly from
mexCallMATLAB(1, &Aout, 3, rhs, "conv2");
to
mexCallMATLAB(1, Aout, 3, rhs, "conv2");
Compiling this, and calling myConv(magic(4)) works without crashing, but you will need to verify that the correct calculations are being performed.
As well, there are a number of place in the code where you are creating matrices (numeric or string) and not freeing the memory. The following lines should be added to Reduce
// destroy arrays
mxDestroyArray(maskTranspose);
mxDestroyArray(mask);
mxDestroyArray(BConvolved);
mxDestroyArray(Temp);
mxDestroyArray(TempConvolved);
and the following line added to convolve
// destroy arrays
mxDestroyArray(rhs[2]);
I've attached a version of the code that compiles (on my system anyway).
