Load multiple csv files and do iterative calculation for each file

Question

I am trying to read multiple csv files and want to plot them all together, 
Lets say I have 3 files, file1 , file2 and file3
I want to average 3 files, and plot them all together, 
So far I have set up like this:

SetDirectory[NotebookDirectory[]]

and I load files like this:

files = FileNames[NotebookDirectory[] <> "*.csv"];
Do[Print["Processing file ", files[[n]]];
 processFile[files[[n]]], {n, 1, Length[files]}]

processFile[fileName_]:=
 Module[{data2},
  data2 = Import[fileName, "Data"];
  tables = Table[data2[[i]], {i, 1, Length[data2]}]]

now, here is where I have problem:
I am giving a name for each processFile and do other calculations:

d1 = processFile["file1.csv"];
d2 = processFile["file2.csv"];
d3 = processFile["file3.csv"];
avg = (d1 + d2 + d3)/3;
avgplot = ListPlot[avg, PlotRange -> {{0, 100}, {0, 1}}];

I am having trouble automating d1,d2,d3 (I want this automated to no matter how many files I have and so I don't have to manually insert their names and define multiple times)

Thanks for your help.

dwa · Answer

What do your CSV files contain? Single columns? Multiple columns?

In any case ...

Generate some data, then export it:

test = RandomReal[{-1, 1}, {4, 1024}];
test = Transpose[{Range[1024], #}] & /@ test;
Export["file_" <> IntegerString[#, 10, 2] <> ".csv", test[[#]]] & /@ 
  Range[Length@test];

Using your function with a small correction to restrict the scope of tables:

processFile[fileName_] := Module[
  {data2, tables},
  data2 = Import[fileName, "Data"];
  tables = Table[data2[[i]], {i, 1, Length[data2]}]
  ]

We can import the data & plot it:

ifile = FileNames["*.csv"];
proc = processFile[#] & /@ ifile;
ListPlot[proc, PlotStyle -> {Black, Gray, Red, Blue}]

But you want to take the average. Lets do that for the first three files & plot the result:

avg = proc[[1, All, 2]] + proc[[2, All, 2]] + proc[[3, All, 2]];
avg = Transpose[{proc[[1, All, 1]], avg/3.}];
ListPlot[tavg]

We can also average all the imported data:

tavg = Mean@proc[[All, All, 2]];
tavg = Transpose[{proc[[1, All, 1]], tavg}];

But perhaps a more elegant way is:

idat = Import[#] & /@ ifile;
iavg = Mean[idat];

which imports the data into a single 4 x 1024 x 2 array (idat), then averages those data.

Load multiple csv files and do iterative calculation for each file

One Answer

Add your own answers!

Ask a Question