Finding data on specific point from large set data

I have a folder named 'THETHA'. Inside folder I have 360 csv file named '1.csv', '2.csv',...'360.csv'. Each csv file has a data of temperature for different location (X ,Y, Z ,T). Each csv file has different number of rows because result is generated from comsol. I want to estimate temperature on specific point in each csv file. I have made another folder named "DATA" which contains my specific point(X, Y ,Z) in which i want to estimate temperature in each csv file. It is possible that our specific point may not be in csv file so i need to find temperature on that point with the help of interpolation. I am attaching my 'DATA' file and one csv file. Please help me.

Answer 1

[AndresVar]

1. Import data (e.g. right click file -> Import data... -> generate code)
2. Use scattered interpolant: Interpolating Scattered Data - MATLAB & Simulink (mathworks.com)

clear;

%%% get XYZ locations to interpolate
opts = delimitedTextImportOptions("NumVariables", 3);
opts.DataLines = [1, Inf];
opts.Delimiter = ",";
opts.VariableNames = ["VarName1", "VarName2", "VarName3"];
opts.VariableTypes = ["double", "double", "double"];
opts.ExtraColumnsRule = "ignore";
opts.EmptyLineRule = "read";
data = readtable("DATA.csv", opts);

XYZq = table2array(data);

%%% import data from a Theta file
clearvars opts;
opts = delimitedTextImportOptions("NumVariables", 4);
opts.DataLines = [10, Inf];
opts.Delimiter = ",";
opts.VariableNames = ["X", "Y", "Z", "TKThetha1"];
opts.VariableTypes = ["double", "double", "double", "double"];
opts.ExtraColumnsRule = "ignore";
opts.EmptyLineRule = "read";
data = readtable("1.csv", opts);
data = table2array(data);

% create a scattered interpolant object
T_data = scatteredInterpolant(data(:,1:3),data(:,4));

% interpolate the values
T_interp = T_data(XYZq)

[AMAN GUPTA]

I am getting error while running this code.Please help me to resolve this error

[AndresVar]

@AMAN GUPTA that function maybe not be available on matlab 2018. Anyway use either:

• csvread instead: (Not recommended) Read comma-separated value (CSV) file - MATLAB csvread (mathworks.com)
• or use the import tool, and generate the code: Import data from file - MATLAB (mathworks.com)

[AndresVar]

@AMAN GUPTA using csvread:

clear;
XYZq = csvread("DATA.csv");
data = csvread("1.csv",9);

% create a scattered interpolant object
T_data = scatteredInterpolant(data(:,1:3),data(:,4));

% interpolate the values
T_interp = T_data(XYZq)

[AMAN GUPTA]

Thank you so much @AndresVar. Your code was very helpful.

[AMAN GUPTA]

I am able to generate my temperature data for 1.csv file by using your recent code. Now i want to generate the same for 360 csv files. Could you please help me to write a loop for that?

[AndresVar]

maybe something like this

XYZq = csvread("DATA.csv");

thetas = 1:360;
T_interp = zeros(size(XYZq,1),numel(thetas)) % a column for each theta

for ii = 1:numel(thetas)
    theta = thetas(ii);
    thetaFilename = sprintf("%d.csv",theta)
    
    data = csvread(thetaFilename,9)
    T_data = scatteredInterpolant(data(:,1:3),data(:,4));
    
    T_interp(:,ii) = T_data(XYZq);
end

[AMAN GUPTA]

Thank you so much @AndresVar.

[AMAN GUPTA]

I am trying this latest code for my another 1001 csv file names as '1.csv','2.csv',...'1001.csv'. But I am getting error and all the element of 'T_interp' are zeros from column no 194. Please help me to resolve this issue.

[AndresVar]

you should verify 194.csv file exist and that the data on it starts at row 10 like the others

debug file 194 outside the loop. You can see the error is line 10, so it has problems reading that file.

[AMAN GUPTA]

@AndresVar can you please help me for some modification?

now file name changes as '1_1.csv', '1_2.csv', ...'1_41'.csv, '2_1.csv', '2_2.csv', ..'2_41.csv',............'71_1.csv', '71_2.csv',......'71_41.csv'.

now how can i modify this code to get my desire data?.

your previous help was too much helpful so please help me to tackle this problem

[AndresVar]

what do the numbers represent?

Anyway, here is one way

XYZq = csvread("DATA.csv");
num1s = 1:71;
num2s = 1:41;
T_interp = nan(size(XYZq,1),numel(num1s),numel(num2s)) % 3 dimensions
for num1 = num1s
    for num2 = num2s
        dataFilename = sprintf("%d_%d.csv",num1,num2)
        data = csvread(dataFilename,9)
        T_data = scatteredInterpolant(data(:,1:3),data(:,4));
        
        T_interp(:,num1,num2) = T_data(XYZq);
    end
end

The alternative is you put the csv files in a folder and read all. for each file make a variable name and assignit in a table or struct. This would be useful if you don't know num1s and num2s

[AMAN GUPTA]

Thank you so much @AndresVar

Actually I have extracted these csv files from COMSOL Multiphysics. I have 2 variable parameter angle and radius. I have guess 41 value of angle and 71 value of radius so will get total 41*71=2911 combination. for each combination I have generated Temperature profile in COMSOL Multiphysics. so for each value of guess radius there will be 41 value of angle so I have given the name like this.

In this code I am facing one problem that my 41*71=2911 Temperature Profile is not in the form of Excel sheet. Can you please help me @AndresVar so that I can get my 2911 temperature profile in the form of excel data and i can use this data further in my calculation

Answer 2

[AMAN GUPTA]

@AndresVar can you please help me for some modification?

now file name changes as '1_1_1.csv', '1_1_2.csv', ...'1_1_11'.csv, '1_2_1.csv', '1_2_2.csv', ..'1_2_11.csv',............'1_9_11.csv', '21_1_1.csv', '21_1_2.csv',.. '21_1_11.csv', '21_2_1.csv', '21_2_2.csv', '21_2_11.csv'.......'21_9_1.csv','21_9_2.csv', '21_9_11.csv'.

now how can i modify this code to get my desire data?.

Please help me.

[AMAN GUPTA]

@AndresVar can you please help me for the problem discussed above

[AndresVar]

you can do the same as before but now 4 dimensions.