Hi Lua users,
I'm trying to read a group of CSV files in the following manner.I have list
of huge log files( about 10-20 Mb stored as CSV) and the filenames are in a
timestamped serial order inside a directory like
logfile1.csv,logfile2.csv,logfile3.csv ......... logfileN.csv .Inside the
logfiles there is a field called TIMESTAMP and I want to combine the
logfiles according to the timestamps.For example a TIMESTAMP(Linux
timestamps are the number of seconds since midnight, January 1, 1970 GMT)
value is 2 in logfile2.csv and 15 in logfile15.csv. So I want to combine all
the logfiles from logfile2.csv to logfile15.csv into one logfile.csv.
More detailed explanation:
I need to do some regular expression matching before combining all the
files(namely matching the timestamps).
So i need to develop a program which can work at the command prompt as :
$combinelogs timestamp1 timestamp2 logfile1.csv logfileN.csv outlogfile.csv
This would search the timestamp1 inside logfile1.csv and write all the
records after a match into outlogfile.csv and then write all the records
from logfile2.csv ,logfile3.csv ........... till the timestamp2 inside
logfileN.csv .
The Linux timestamps(e.g 1166212618.66 ) are the first fields in each record
in the logfiles and to be provided by the user at the command prompt.
Please give some suggestions for an efficient way to do this since the log
files are pretty large.I hope Lua can do this pretty efficiently.I'm on Suse
Linux.
--
Regards,
Chandan
