Issue concatenating .tdms files

I am acquiring data from Load Cells, Thermocouples and Analog Voltages using NI Signal Express. The Load Cells/DAQ chain require a high sample rate so I am forced to take multiple 1612Hz 1s logs every minute as opposed to low sample rate over the 2.5hours of logging.

Signal Express outputs a .tdms file for each of LC's, TC's and AI's. The .tdms file also unfortunately contains 6 additional downsampled Channel Groups for each log which I haven't found a way to eliminate. The tdmsreader outputs a 1x1050 cell array which I then pick the relevant data out of.

The code below works for 10 logs (few minutes execution), however when I have tried with the 150 log files for 2, and then 20 hours and terminated the execution it only successfully concatenates the Load Cell groups.

There are 8 TC channels and only 2 Load Cells. I initially tried to concatenate the whole dataset, and then again tried to concatenate the heavily downsampled (2 samples) Channel Groups, both with the same result of only the Load Cell data being concatenated.

fileName1 = "Custom_Voltage.tdms";

info = tdmsinfo(fileName1);

a = info.ChannelList;

b = max(a(:,1));

b = b{:,:};

numlogs = b/7;

% Load Cells

data1 = tdmsread("Custom_Voltage.tdms");

n = [1: 7 : (7*numlogs -6)];

loadcells = cat(1,data1{n});

pinload = loadcells(:,1);

pinload = pinload{:,:};

plungerload = loadcells(:,2);

plungerload = plungerload{:,:};

% Temperatures

data2 = tdmsread("Thermocouple.tdms");

thermocouples = cat(1,data2{n});

pintemp = thermocouples(:,5);

pintemp = pintemp{:,:};

The other option for exporting the data from Signal Express is to create 150 individual logs but since the single file concatenation worked for 10 logs I assumed it would for 150.

I’m a beginner user so any help is greatly appreciated.

Answer 1

[Sugandhi]

Hi,

I understand that you are experiencing issues with concatenating the data from multiple TDMS-files acquired using NI Signal Express. The code you provided appears to be working for a smaller number of logs but fails when trying to concatenate a larger number of files.

One potential reason for this issue could be memory limitations. As you concatenate more logs, the size of the resulting data arrays increases, which can consume a significant amount of memory. If the available memory is not sufficient, it can lead to the termination of the execution or incomplete concatenation.

To address this issue, you can try the following suggestions:

1. Increase memory allocation: Check if your MATLAB environment has enough memory allocated to handle the concatenation of a larger number of logs. You can adjust the memory allocation by going to the MATLAB Preferences and modifying the 'Java Heap Memory' or 'Workspace' settings. Increasing the memory allocation may allow you to concatenate a larger number of logs successfully.

2. Concatenate in smaller batches: Instead of concatenating all logs at once, you can try concatenating them in smaller batches. For example, you can concatenate logs in groups of 10 or 20 and save the intermediate results. This approach can help manage memory usage and reduce the likelihood of hitting memory limitations.

The down sampled Channel Groups in the TDMS-files may be causing issues during concatenation. You can try removing or excluding these down sampled Channel Groups from your concatenation process to see if it helps resolve the issue.