Captain
Captain
320 views

syslog connector multiline regex parse problem

Hello all!

I'm developing a parser for logs which contains multiline and single line records aswell. It catches the multline records finely as long as i use the 'regex=(.*)' statement, but with it, this parser also captures logs it shouldnt. There are about ~10 different devices that would send logs to this connector, i could separate them by submessages, but it came to discussion that there will be linux audit logs through syslog, and since i have the (.*) regex, the parser captures it instead of a default parser.

The relevant parts of the parser:

# FlexAgent Regex Configuration File
multiline.starts.regex=^\\d{4}-\\d{2}-\\d{2}\\s\\d{2}:\\d{2}:\\d{2}.\\d{3}\\s\\s\\S+.*
#multiline.singleline.nowaiting=True
multiline.ends.regex=^\\}$
do.unparsed.events=False

regex=(.*)

token.count=1

token[0].name=Message
token[0].type=String

 

submessage[0].pattern.count=11
submessage[0].pattern[0].regex=(\\d+\\-\\d+\\-\\d+\\s\\d\\d\:\\d\\d\:\\d\\d\\.\\d+) .*?(\\S+) (\\d+)(.*?)\\[(.*?)\\]\\s(.*?)\:(.*)
submessage[0].pattern[0].fields=event.deviceReceiptTime,event.deviceSeverity,event.name,event.message,event.deviceCustomString3
submessage[0].pattern[0].names=$1,$2,$6,$7,$5
submessage[0].pattern[0].mappings=$1|$2|$6|$7|$5
submessage[0].pattern[0].types=TimeStamp,String,String,String,String
submessage[0].pattern[0].formats=yyyy-MM-dd HH:mm:ss.SSS,null,null,null,null

The multiline event:

2019-11-05 12:51:44.758 INFO 1 --- [processname] component : data: {
"0": {
"someId": 5,
"someMoreId": "201",
"someVersion": "3.5"
},
"1": {
"someId": 1,
"someMoreId": "2033",
"someVersion": "2"
},
"2": {
"someId": 5,
"someMoreId": "7564",
"someVersion": "1"
}
}

PS: i also tried to modify the agent.properties, and put the generic_syslog before the flexagent_syslog
agents[0].customsubagentlist=cef_syslog|linux_auditd_syslog|generic_syslog|flexagent_syslog

0 Likes
2 Replies
Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

Try to remove the  caret ^ symbol from your multiline.starts.regex and multiline.end.regex

0 Likes
Highlighted
Captain
Captain

i've already tried every format of the regex, it's the same
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.