Highlighted
New Member.

Re: Logger Archive Extractions

sounds good.  I'll stick to 2.6.  I'll get around to sending a pull request later tonight or tomorrow.  Many thanks from all of us for submitting it.

You and Tracy rock for getting this out there!

0 Likes
Reply
Highlighted
Super Contributor.
Super Contributor.

Re: Logger Archive Extractions

Any news on turning CEF to JSON ? Did you managed to create some code for that ?

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: Logger Archive Extractions

This exists today. Just use the -j option.

0 Likes
Reply
Highlighted
Established Member..
Established Member..

Re: Logger Archive Extractions

Thanks for sharing your updates on github.

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: Logger Archive Extractions

That's weird but the script did not work with archives were created by Logger 5.5. I find out that records delimiters were different for each archive. As the result I created a modification of  the script that just looking for "CEF" records inside and it ignores all internal headers structure ("quick and dirty" way). It works well with few Tbs of archives. Unfortunately I had to cut all JSON capability since RHEL 5.x still has Python 2.4.

Also I created a batch script that handles number of archives. What is weird - ## in the Data file has leading zero but the corresponding CSV does not have it in the ##. The XML file does not have names of CSV files.

Thank you for the great script!

Regards,
Alex.

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: Logger Archive Extractions

Hello All,

Has this ever been made to work on Windows?

I've installed Python for Windows from www.python.org (v3.4.2), the initial run of the script complained about print not having parenthesis, poped those in there, then it complained elsewhere:

D:\Python34\Scripts>lacat-win.py ArcSight_Data_1_0504403158265495879.dat ArcSight_Metadata_1_504403158265495879.csv.gz

Traceback (most recent call last):

  File "D:\Python34\Scripts\lacat-win.py", line 169, in <module>

    dump_cef(f_dat, f_csv, to_json=options.json, limit_to=options.filter)

  File "D:\Python34\Scripts\lacat-win.py", line 140, in dump_cef

    for chunk in get_metadata(open(fcsv)):

  File "D:\Python34\Scripts\lacat-win.py", line 66, in get_metadata

    rows = [r for r in csv.reader(f)]

  File "D:\Python34\Scripts\lacat-win.py", line 66, in <listcomp>

    rows = [r for r in csv.reader(f)]

  File "D:\Python34\lib\encodings\cp1252.py", line 23, in decode

    return codecs.charmap_decode(input,self.errors,decoding_table)[0]

UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 72: character maps to <undefined>

J-

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: Logger Archive Extractions

Have you tried it with Python 2.4/2.7? Some Python libraries were significantly changed in 3.*

0 Likes
Reply
Highlighted
Respected Contributor.
Respected Contributor.

Re: Logger Archive Extractions

Hi Jeff,

I see something similar on RHEL, if I don't gunzip the Metadata-archive. Looks like the script isn't able to "look into" the gz-file.

It would be great to hear, if anyone had found a solution to this issue, so we wouldn't have to unpack every file (we are going to migrate about a years event archives )

/Claus

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: Logger Archive Extractions

This is a great python script!  Has anyone written one that can work on ESM Archives?

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: Logger Archive Extractions

Why i can not use categoryBehavior for export? and What's command for OR Condition?

./lacat -f dvc=10.0.208.16 -s categoryBehavior=/Access ArcSight_Data_2_0504403158265496184.dat ArcSight_Metadata_2_504403158265496184.csv > /tmp/test3.csv

Usage: lacat [options] path_to_dat path_to_meta

Extracts cef events from Logger Archive files to stdout

THIS SOFTWARE IS UNSUPPORTED.  USE AT YOUR OWN RISK.

Why is it called lacat?

    Because "Logger_Archive_cat" was too long to type.

lacat: error: no such option: -s

0 Likes
Reply
Highlighted
Honored Contributor.. Honored Contributor..
Honored Contributor..

Re: Logger Archive Extractions

Does anyone know the directory in which Logger stores the events?  I think it is /opt/data/logger but not sure.  Thanks!

0 Likes
Reply
Highlighted
Outstanding Contributor.
Outstanding Contributor.

Re: Logger Archive Extractions

Hi Jeremy,

Does this support newer versions of logger as well?

0 Likes
Reply
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.