Analyse and make pattern from csv file

Hi,

I need to do some data mining from a eventlog.CSV file,use the correlation function to analyse and pattern it. So i need the algorithm ho to achieve it. below is the pattern of my log file

ullid;date;time;EventType;Event 00000000000000023931;2012-08-09;17:58:59;Status;zeroSds 00000000000000023946;2012-08-09;17:58:59;Status;configuring 00000000000000071571;2012-08-09;17:59:23;Status;idle 00000000000000071655;2012-08-09;17:59:23;Status;last ShutDownMoment: 2012/08/09 13:40:35 00000000000000071657;2012-08-09;17:59:23;Status;last StartUpMoment: 2012/08/09 17:25:31 00000000000000071659;2012-08-09;17:59:23;Status;current StartUpMoment: 2012/08/09 18:04:51 00000000000000071661;2012-08-09;17:59:23;Status;initiating 00000000000000139036;2012-08-09;18:10:14;Error;reportErrorId(MRE): correctStop * (/embeddedControl: disconnected) * 00000000000000147300;2012-08-09;18:11:21;Error;reportErrorId(MRE): correctStop * (/embeddedControl: dppConnectionBroken) * 00000000000000147724;2012-08-09;18:11:22;Error;reportErrorId(MRE): correctStop * (/engineFunction/process/printTransport/deviceIntermTransMo: positionError) * 00000000000000148044;2012-08-09;18:11:22;Error;reportErrorId(MRE): correctStop * (/engineFunction/process/printTransport/deviceSheetInputInpMo: positionError) * 00000000000000148361;2012-08-09;18:11:22;Error;reportErrorId(MRE): correctStop * (/engineFunction/process/printTransport/deviceSheetInputOutMo: positionError) * 00000000000000148394;2012-08-09;18:11:22;Warning;reportErrorId(Service Warning): noStop * (/paperInputPIM1LeafFunction: maxMotorPositionError) * 00000000000000148423;2012-08-09;18:11:23;Warning;reportErrorId(Service Warning): noStop * (/paperInputPIM1LeafFunction: speedAirSupplyMotorIncorrect) * 00000000000000148441;2012-08-09;18:11:23;Error;reportErrorId(MRE): correctStop * (/embeddedControl: resetBeltDrive) * 00000000000000148457;2012-08-09;18:11:23;Error;reportErrorId(MRE): correctStop * (/embeddedControl: printHeadGapControl) * 00000000000000148473;2012-08-09;18:11:23;Error;reportErrorId(MRE): correctStop * (/embeddedControl: resetBeltIntermediateTransport) * 00000000000000148488;2012-08-09;18:11:23;Error;reportErrorId(MRE): correctStop * (/embeddedControl: resetInputDuplex) * 00000000000000148503;2012-08-09;18:11:23;Error;reportErrorId(MRE): correctStop * (/embeddedControl: resetInputCool) *

thanks in advance

[Cedric]

What is it that you want to extract from this file? If it is always located at the same place/column in each row, you can use usual functions for extracting formatted content from text files; otherwise you can use regular expressions to match patterns for your data extraction.

[Ajay Selvam]

extract only the error.the errors are in the same column but not the same place,different places for each error.

Answer 1

[Cedric]

You will have to define a little better what you want to extract when errors are logged, but here is an example using REGEXP:

 >> buffer = fileread('events.log') ;
 >> results = regexp(buffer, '(?<date>.{10});(?<time>.{8})(?=;Error)', 'names')
 results = 
 1x10 struct array with fields:
    date
    time

 >> results(1)
 ans = 
    date: '2012-08-09'
    time: '18:10:14'

Using a more standard approach and the assumption that events always start at character #42, you can do something like:

 fid = fopen('events.log') ;
 while ~feof(fid)
    line = fgetl(fid) ;
    if numel(line) < 42 || line(42) ~= 'E',  continue ;  end

    % This is an error line; do something, e.g extract date and time.

 end
 fclose(fid) ;