Highlighted
Valued Contributor.
Valued Contributor.
623 views

FlexParser doesn't pick all of the matched event

Hi All,

I've built a flexagent parser with regex, it works well mostly, but I see there are some unparsed events that exactly should fit the regex, but the parser doesn't pick, altough the others picked perfectly. This semi-working rate is less than 10%, 90% is fine. Checked several times, the unmatched logs is pretty much the same as the others.

What can cause this problem? 

Thanks in advance!

 

0 Likes
6 Replies
Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

To be able to provide a sufficient answer we would really need to see a copy of a log entry that works, one that does not work, and the regex you use in the parser, as there is no other real reason as to why this behaviour should happen.

Feel free to censor any information you think is sensitive, but please keep the format exactly the same, no extra spaces or characters (so switching out a number with another number is fine, but not with a letter etc)

You could also look in agent.log for information about parsing errors, it might point you towards the root cause of the issue.

-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
0 Likes
Highlighted
Respected Contributor.. Respected Contributor..
Respected Contributor..

Hi,
Is there any empty tokens value in your logs ? That might cause a shifting in the token list and might lead error during parsing process.

0 Likes
Highlighted
Valued Contributor.
Valued Contributor.

Thanks for your help, I've just diggin in the forum about possible causes, and I think I am in a parsing order trouble.

I've written a parser for a custom device that sends more daemon logs incl pf, iked, sshd, etc. with a very crap, unhandled format. The parser that was written is only to handle the pf daemon logs but not the other. So the device is always the same but the logformat is different based on its daemon. I beleive every time when the other type of logs (iked, sshd) arriving, these are picked up by the default generic syslog parser, -> placed on the syslog.properties, override my fine parser to the generic one and it takes some time the device overriden back to the correct flexagent parser when pf format log arrives again. And thats circulating from time to time. 

So the question is, how could I force the agent to use the custom parser only, and not pick the others daemon logs by the default generic.

This is the thread that is similar to my case.

https://community.softwaregrp.com/t5/ArcSight-User-Discussions/syslog-parsing-order-subagents/td-p/1558898

 

Thanks!

0 Likes
Highlighted
Respected Contributor.
Respected Contributor.

You can try move your subpparser (something like yourparsername_syslog) in agent.properties > agent[0]customsubagentlist to beginning of the list.

Then remove syslog.properties file and restart parser.

0 Likes
Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

agents[0].usecustomsubagentlist=true

agents[0].customsubagentlist=flexagent_syslog

and for details :

"Boy Meets ArcSight" or "the fairy tales of a ill-treated ArcSight admin" --- Epsiode 1.0 - custom syslog parser ---

0 Likes
Highlighted
Super Contributor.
Super Contributor.

This wouldn't work. I have created service request to this issue. Order of the parsers in configuration file doesn't have any impact in real order of parsers. You can only drop parsers from configuration. I'm not sure, if it is possible to drop default/generic parser at all.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.