Fred11 Absent Member.
Absent Member.
346 views

How can we pars pars same line several time in flexconnector

Jump to solution

Hi Everybody,


I need help to find a way to parse a line within logfile several time in one regex for one flexconnector, How is it possible? let me explain. I have a file which i need to parse and rows have not the same information but i want to take out some part of the line if that information and part exist. and then get back and parse again the line for another part and check if there is some. 

Just think that my log rows are as below

#a=A b=B c=C d=D f=F

want with regex in flexconnector to be able to parse it several time like this it goes first and take out the just one value (ex. a and regex like (a=\w .*)) and gets back and goes again through the line and takes out Another like (.*b=\w.*)  But if there is not any value (like (.*e=\w.*)) just go further untill all regex finishes and parse all of the value which exist.

I hope I could explain what I am looking for and thank you for your time

Best//Fred


Labels (2)
0 Likes
1 Solution

Accepted Solutions
Highlighted
kitdaddio Absent Member.
Absent Member.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

Hi,

(I didn't test the following examples. Sorry if it doesn't work.)

You can grab the whole line into a string (Msg) (or grab the part that you want to test multiple times) and use individual regex commands to parse it.

In this example, the fields are space-delimited and name=value layout, so I start off with "\\s" before the field name.

regex=^(.*)$

token.count=1

token[0].name=Msg

token[0].type=String

event.destinationHostName=__regexToken(Msg,\\sdhost=(\\S*))

event.sourceAddress=__regexTokenAsAddress(Msg,\\ssrc=(\\S*))

event.deviceCustomString1=__concatenate("externalId:",__regexToken(Msg,\\sexternalId=(\\S*)))

event.agentReceiptTime=__safeToDate(__regexToken(Msg,\\sart=(\\S*)))

For conditional mapping, you can use the following. In this made-up example, the condition is destinationHostName="host1", in which case it looks for a classId field:

event.deviceCustomString2=event.destinationHostName

#Conditional mappings

conditionalmap.count=1

conditionalmap[0].mappings.count=1

conditionalmap[0].field=event.deviceCustomString2

conditionalmap[0].mappings[0].values=host1

conditionalmap[0].mappings[0].event.deviceEventClassId=__regexToken(Msg,\\sclassId=(\\S*))

I think you could also use the __ifThenElse:

event.deviceEventClassId=__ifThenElse(__stringTrim(event.destinationHostName),null,stringConstant("UNKNOWN"),__regexToken(Msg,\\sexternalId=(\\S*)))

Or you can use that to set a binary variable value, and then do __ifPositive(BinaryVariable

Have fun

Kit Lueder

0 Likes
13 Replies
rkent1 Acclaimed Contributor.
Acclaimed Contributor.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

Hi Fred,

The approach of 'parsing the same line multiple times' doesn't sound like the right approach in my opinion. The outcome you are seeking should be accomplishable using regex techniques, perhaps lookaheads are what you need to use (Regex Tutorial - Lookahead and Lookbehind Zero-Length Assertions)

Can you give examples of two or more messages, and an example of what output mappings you are trying to achieve?

0 Likes
Highlighted
kitdaddio Absent Member.
Absent Member.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

Hi,

(I didn't test the following examples. Sorry if it doesn't work.)

You can grab the whole line into a string (Msg) (or grab the part that you want to test multiple times) and use individual regex commands to parse it.

In this example, the fields are space-delimited and name=value layout, so I start off with "\\s" before the field name.

regex=^(.*)$

token.count=1

token[0].name=Msg

token[0].type=String

event.destinationHostName=__regexToken(Msg,\\sdhost=(\\S*))

event.sourceAddress=__regexTokenAsAddress(Msg,\\ssrc=(\\S*))

event.deviceCustomString1=__concatenate("externalId:",__regexToken(Msg,\\sexternalId=(\\S*)))

event.agentReceiptTime=__safeToDate(__regexToken(Msg,\\sart=(\\S*)))

For conditional mapping, you can use the following. In this made-up example, the condition is destinationHostName="host1", in which case it looks for a classId field:

event.deviceCustomString2=event.destinationHostName

#Conditional mappings

conditionalmap.count=1

conditionalmap[0].mappings.count=1

conditionalmap[0].field=event.deviceCustomString2

conditionalmap[0].mappings[0].values=host1

conditionalmap[0].mappings[0].event.deviceEventClassId=__regexToken(Msg,\\sclassId=(\\S*))

I think you could also use the __ifThenElse:

event.deviceEventClassId=__ifThenElse(__stringTrim(event.destinationHostName),null,stringConstant("UNKNOWN"),__regexToken(Msg,\\sexternalId=(\\S*)))

Or you can use that to set a binary variable value, and then do __ifPositive(BinaryVariable

Have fun

Kit Lueder

0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

Re: How can we pars pars same line several time in flexconnector

Jump to solution

Base on your input:

Just think that my log rows are as below

#a=A b=B c=C d=D f=F

In your log is it always a=something b=something etc?  What I want to know if its like tokens with value?

------------------------------------
Please use the Like button below, if you find this post useful or mark it as an accepted solution if it resolves your issue.
0 Likes
Fred11 Absent Member.
Absent Member.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

Hi Eric,

Yes they are, just as I mentioned they come in the rows differently like below:

#a=A b=B c=C d=D f=F

#a=A c=C d=D f=F

#b=B c=C d=D e=E


and as you said a, b, c, d, e and f are always know as a known text but the values chang.


0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

Re: How can we pars pars same line several time in flexconnector

Jump to solution

Do you have a lot of possibilities? and also are the value always the same for the token (a,b,c etc)?

------------------------------------
Please use the Like button below, if you find this post useful or mark it as an accepted solution if it resolves your issue.
0 Likes
rkent1 Acclaimed Contributor.
Acclaimed Contributor.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

Fred,

Again, I'm not sure if I fully understand the details, but a regex with a lot of conditionals could do the trick here. Have a look at this regex mockup:

http://regex101.com/r/oL1zV9/3

It matches all 3 test strings. Let me know if there is some extra detail that I am missing if this doesn't line up with what you want. Remember with regexes and parsing in general, the devil is in the details.

0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

Re: How can we pars pars same line several time in flexconnector

Jump to solution

There is a easier solution but I need to have more info, as my previous post.

------------------------------------
Please use the Like button below, if you find this post useful or mark it as an accepted solution if it resolves your issue.
0 Likes
Fred11 Absent Member.
Absent Member.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

Hi Eric,

Have a lot of possibilities! a, b..... have and can get a lot of different values.

0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

Re: How can we pars pars same line several time in flexconnector

Jump to solution

If you know every possibilities you can use the extraprocessor in your flex to treat each possibilities.  If you want an example let me know.

------------------------------------
Please use the Like button below, if you find this post useful or mark it as an accepted solution if it resolves your issue.
0 Likes
Fred11 Absent Member.
Absent Member.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

No! these are the values that i am intrested in to pull out and map to schema in ESM. I appreciate any example which help me to go further.

0 Likes
Fred11 Absent Member.
Absent Member.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

Thank you Christopher,

0 Likes
kitdaddio Absent Member.
Absent Member.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

I think that one glitch of that regex is the fields are marked optional ("?" tag) but the space delimiters are not marked optional, so this would fail parsing if there wasn't a double-space after the first term, since the b= term is omitted:

#a=A c=C d=D f=F

It also doesn't handle the fields being in different order (which my example at top handles, since each regex action is independent).


Note , ti would through the entire expression in the return (a=A), not just the value (A), so you would need additional parsing to separate the a from the A.

Kit.


0 Likes
kitdaddio Absent Member.
Absent Member.

Re: How can we pars pars same line several time in flexconnector

Jump to solution

FYI, related topic -- see my message for how to process AD logs into CEF format.

It shows how to use the extraprocessor to process the key=value (space delimited) format.

https://protect724.hp.com/message/53059#53059

Kit.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.