Highlighted
zargaran Honored Contributor.
Honored Contributor.
1046 views

How can i create parser for java application log?

Jump to solution

Dear All

I have a sample log from java application with SSO authentication contents.

but in this file i have many log line that i does not needed for indexing in parser. 

how can i except the selection of log line in parser flex connector creation?

BR

Amir

Labels (1)
0 Likes
1 Solution

Accepted Solutions
Acclaimed Contributor.. Shaun Acclaimed Contributor..
Acclaimed Contributor..

Re: How can i create parser for java application log?

Jump to solution

Check the Flexconnector Developer Guide, but I believe you can use one of the following:

line.include.regex

line.exclude.regex

View solution in original post

3 Replies
Ubaid Valued Contributor.
Valued Contributor.

Re: How can i create parser for java application log?

Jump to solution

Hi, 

 

It should not be problem, you can only parse the events/lines you need, leave rest or put them in an all logs container and igonre it. 

 

Regards

Ubaid 

0 Likes
Acclaimed Contributor.. Shaun Acclaimed Contributor..
Acclaimed Contributor..

Re: How can i create parser for java application log?

Jump to solution

Check the Flexconnector Developer Guide, but I believe you can use one of the following:

line.include.regex

line.exclude.regex

View solution in original post

zargaran Honored Contributor.
Honored Contributor.

Re: How can i create parser for java application log?

Jump to solution

dear @Shaun

can i have a multiple line.exclude.regex ?

if yes ,how can i put all exclusion lines on the parser?

 

Parser : 

do.unparsed.events=true

#################################################################################
############# Included and Excluded Log lines ###################
#################################################################################

line.include.regex=^\w\w\w \d\d \d\d:\d\d:\d\d \S+ [Default Executor-thread-\d+] ROOT.*
line.exclude.regex=^\w\w\w \d\d \d\d:\d\d:\d\d \S+ \#.*
line.exclude.regex=^\w\w\w \d\d \d\d:\d\d:\d\d \S+ [Default Executor-thread-\d+] ir.*
line.exclude.regex=^\w\w\w \d\d \d\d:\d\d:\d\d localhost.*
line.exclude.regex=^\w\w\w \d\d \d\d:\d\d:\d\d \S+ [DefaultQuartzScheduler.*
line.exclude.regex=^\w\w\w \d\d \d\d:\d\d:\d\d \S+ tag_audit_log.*

#################################################################################
################# Main Regex ################
#################################################################################

regex=(\\S+\\s+\\d+ \\d\\d\:\\d\\d\:\\d\\d) (\\S+) \\[Default Executor\\-thread\\-(\\d+)\\] ROOT \\{"ip"\:"(\\d+\\.\\d+\\.\\d+\\.\\d+)","(.*"successful"\:"([true|false]+)".*)"}

#################################################################################
################### Tokens ####################
#################################################################################
token.count=6

token[0].name=date
token[0].type=TimeStamp
token[0].format=MMM dd HH\:mm\:ss

token[1].name=server
token[1].type=String

token[2].name=code
token[2].type=Integer

token[3].name=ip
token[3].type=IPAddress

token[4].name=Message
token[4].type=String

token[5].name=id
token[5].type=String


submessage.messageid.token=id
submessage.token=Message


#################################################################################
################### Mapping ###################
#################################################################################

event.message=Message
event.deviceHostName=server
event.deviceEventClassId=id
event.sourceAddress=ip
event.endTime=date
event.sessionId=code

#################################################################################
############### Custome Mapping ###############
#################################################################################
event.deviceCustomString1Label=__stringConstant("Login_Date")
event.deviceVendor=__stringConstant("myProduct")
event.deviceProduct=__stringConstant("myVendor")
event.deviceVersion=__stringConstant("0.0.1")

#################################################################################
############ Severity Mapping ##################
#################################################################################

severity.map.medium.if.deviceSeverity=true
severity.map.high.if.deviceSeverity=false


#l10n.filename.prefix=
#################################################################################
################ Sub Messages ##################
#################################################################################
submessage.count=2

submessage[0].messageid=true
submessage[0].pattern.count=1
submessage[0].pattern[0].regex=loginDate"\:"(.*)","successful"\:"(.*)","username"\:"(\\S+)
submessage[0].pattern[0].fields=event.deviceCustomString1,event.sourceUserName,event.deviceSeverity
submessage[0].pattern[0].mappings=$1|$3|$2

submessage[1].messageid=false
submessage[1].pattern.count=1
submessage[1].pattern[1].regex=loginDate"\:"(.*)","successful"\:"(.*)","username"\:"(\\S+)
submessage[1].pattern[1].fields=event.deviceCustomString1,event.sourceUserName,event.deviceSeverity
submessage[1].pattern[1].mappings=$1|$3|$2

 

if i set in my syslog connector i get this info log in agent.log :

 First event from [|||localhost] received

nothing of included lines does not pars correctly. also all pre-requirments of  syslog flex connector configured correctly (agents[0].customsubagentlist=subagent_syslog , agents[0].usecustomsubagentlist=true) . 

Also be inform that i write "regex" just for "include line". 

where is my missed configuration?

please Help me this is very critical job!

BR

Amir

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.