Last post Sep 17, 2010 06:48 PM by atconway
Sep 13, 2010 02:05 AM|jacobzhang|LINK
It is a very large .txt file (more than 3M), and produced everyday, the content is user's system log like below:
2007-11-01 18:20:42,983  INFO GetXXX() SERVICE START
2007-11-01 18:21:42,983  WARING USER ACCESS DENIED
2007-11-01 18:22:42,983  ERROR INPUT PARAMETER IS NULL CAN NOT CONVERT TO INT32
2007-11-01 18:23:59,968  INFO USER LOGOUT
Any idea? Thanks.
Sep 13, 2010 03:13 AM|SGWellens|LINK
What do you want to do that Notepad or Excel will not do?
Sep 13, 2010 04:35 AM|ignatandrei|LINK
Sep 13, 2010 05:27 AM|gabriel.lozano-moran|LINK
You need a log management strategy, what to log, for how long, where you put the log, who can access it, etcetera. In large companies and even in smaller companies maybe like in your case going manually through 3mb of logfiles on daily basis is a tedious
job. You should work out a buy or build decision matrix or maybe look for a log parser on the open source market instead of manually going through these logfiles.
Sep 13, 2010 05:53 AM|toas1|LINK
I use a log viewer called bare tail from
bare metal. its great for reading and following log files. it can be setup with filters that highlight rows in different colors based on whats in the text. it can also "follow" a log file in realtime and keep scrolling when new rows appear. And its free
to use :)
Sep 13, 2010 05:59 AM|jacobzhang|LINK
I'll save the log into database.
Sep 13, 2010 06:36 AM|gabriel.lozano-moran|LINK
You don't have to save them in a database if thete is no real requirement. A big advantage of working with files is the portability. On the other hand if you have multiple sources and you want to write a custom logger you could opt for database logging.
Sep 16, 2010 04:07 AM|jacobzhang|LINK
Any more ideas?
Sep 16, 2010 06:07 AM|toas1|LINK
Any more ideas?
Did you try bare tail?
Sep 16, 2010 08:20 AM|ignatandrei|LINK
Any more ideas?
Did you try logparser ?
Sep 16, 2010 07:11 PM|Wilmar|LINK
We'll it seems you only need a better way to read and analize the log file. So why not import this into an XML file and take advantage of the XML classes?
Sep 17, 2010 09:37 AM|jacobzhang|LINK
Another software creates the log file, so I can not change to xml formate.
Sep 17, 2010 09:41 AM|jacobzhang|LINK
Hi ignatandrei, LogPaser is powerful, but my client log file formate does not have column header.
Sep 17, 2010 10:16 AM|ignatandrei|LINK
try also http://www.filehelpers.com/
Sep 17, 2010 06:48 PM|atconway|LINK
Honestly if this is a daily requirement to analyze the file (and who knows to what extent), then I too would opt for getting it into a more powerful medium like a SQL database. Now you have almost an infinite amount of tools at hand to 'analyze'
the data: Stored Procedures, Views, .sql files, .NET code including LINQ technologies, ADO.NET, and on and on.... You can write apps around the data to read it and represent to end users in the order they want it. All you need to do is write some code to parse
that files data out and then have a ton of power at your fingertips to analyze it. I don't think moving it to a db is too much overhead or work if this truly is a daily need.
One other idea would be just to write your own code to parse the file using the System.IO namespace in .NET, and re-presenting the data in a more readable manner right on the spot (just import and process). I agree it is difficult to read a text file that
The log tools recommended are a good idea if there is some reason you can't move this file out of its original format.