SparsePack

Packs a sparse matrix into a single uint8 array, reducing memory usage.

This package is particularly useful if you have a lot of relatively small sparse matrices in memory which are not frequently used. By packing the matrices, the memory usage is reduced. As a next step, the result can be compressed with GZIP (as with CompressLib: http://www.mathworks.com/matlabcentral/fileexchange/25656-compression-routines).

Reduction of the size is obtained in 2 ways:
1) By using only the number of bytes required for the sparse pointers. By default, 64-bit Matlab uses 64-bit pointers (8 byte). This is overkill if the dimensions of your matrix are small (e.g. <65000).

2) By offering a different numeric format. Matlab can only store doubles as sparse, while a single or int8 may provide enough precision.

Formats supported are logical/int8/uint8/int16/uint16/int32/uint32/single/double/pattern. 'pattern' is a special format which only stores the sparsity pattern (when unpacked, every non-zero entry will be 1). In contrast, the 'logical' type also remembers if an entry is 0 or not.

Special tags are 'keep' and 'cat'. By default, SparsePack only packs the number of used nonzeros (taken from the index). By specifying 'keep', all entries are stored (see spalloc for more information). 'cat' simply concatenates the vectors without any memory reduction.

Rounding is done the way Matlab does: 139.4 is rounded to 127 in int8 type.
The code is extensively tested with Matlab 2009b on 64-bit Linux, but also compiles and runs well on 32-bit Linux and 32-bit Windows. You can even exchange a packed sparse matrix between 64-bit and 32-bit, and a nice error is given if the unpacked matrix does not fit on the 32-bit system.

The code probably also depends on the little endianess of the byte ordering. I have no big endian hardware to test/optimize it on.

SparsePack is fast and a compression rate of 50% is easily met.

Compile:
mex -largeArrayDims SparsePack.c
mex -largeArrayDims SparseUnPack.c

Test:
testSparsePack

Use (see screenshot):
B = SparsePack(sparse(A), 'single', 'keep');
C = SparseUnPack(B);