ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins. Read more for important details.
ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins.Read more for important details.
674 views

Regex not parsing log file

Hello Guys,

First of all a very Happy New Year to everyone !!!!

Last few days I am tying to build a flex connector for one of our in-house application. The Flex Connector type is "Regex Folder Follower". I have created a configuration file & regex for the log file. But the regex I have created is not working properly. Every time I run the connector it gives me an error:

java.util.regex.PatternSyntaxException: Illegal character range near index 60.

My log file pattern is :

[#| 24 Dec 2013 00:00:25,290 | Portal | INFO | portalServer | PAKUtil.authenticate(user, password) | (002):Connecting to BizLogic server with user XXXXX. | Web-Worker-Thread : 30 |#]

[#| 24 Dec 2013 00:00:25,333 | Portal | INFO | portalServer | PakUtil.authenticate(user, password) | (057):BizSite user XXXXX logged in and the concurrent user count is 1. | Web-Worker-Thread : 30 |#]

[#| 24 Dec 2013 00:00:27,025 | Portal | INFO | portalServer | UserManager.loginUser(request,user,password) | (6119):The user "XXXXX" has logged in. | Web-Worker-Thread : 31 |#]

[#| 24 Dec 2013 00:00:27,065 | Portal | INFO | portalServer | UserManager.loginUser(request,user,password) | (6119):The user "XXXXX" has logged in. | Web-Worker-Thread : 30 |#]

[#| 24 Dec 2013 00:00:33,388 | Portal | INFO | portalServer | UserManager.logoutUser(sessionData) | (6134):Session has been invalidated for user "XXXXX". | Web-Worker-Thread : 30 |#]

And my configuration file looks like:

# FlexAgent Regex Configuration File

regex=\[#\| (\d+) (\S+\s+\d+ \d\d:\d\d:\d\d),(\d+) \| (\w+) \W (\w+) \W+([^-]*)\ Web-Worker-Thread : (\d+).*

token.count=7

token[0].name=Dateoftheevent

token[0].type=Integer

token[1].name=Timeoftheevent

token[1].type=String

token[2].name=id

token[2].type=Integer

token[3].name=Sourceofthelog

token[3].type=String

token[4].name=Typeofthelog

token[4].type=String

token[5].name=Information

token[5].type=String

token[6].name=ThreadNumber

token[6].type=Integer

event.deviceCustomNumber1=Dateoftheevent

event.deviceCustomString1=Timeoftheevent

event.deviceCustomNumber2=id

event.deviceCustomString2=Sourceofthelog

event.deviceCustomString3=Typeofthelog

event.deviceCustomString4=Information

event.deviceCustomNumber3=ThreadNumber

event.deviceVendor=_getVendor("MyVendor")

event.deviceProduct=_stringConstant("MyProduct")

Even I tested the regex on regextester & it gives me perfect match. But somehow my flex connector don't think so.

Anyboday can please chip in some suggestions on this? Pls let me know where am I going wrong?

Thanks.



Labels (2)
0 Likes
6 Replies
Cadet 2nd Class Cadet 2nd Class
Cadet 2nd Class

Your regex isn't using the ArcSight-specific-stuff.  Backslashes should be escaped.  So \w would be \\w in the arcsight world. 

I'd load it up into the arcsight regex tool and let the tool do the work for you.  You can test your regex in the tool and it'll generate a properties file that handles the arcsight gotchas. 

Also, you'll want to break this out into submessages since the log entries aren't the same every time.

0 Likes

Hi Chris,

Thanks for your asistance. I tried using Regex tool but somehow I couldn't figure it out its efficency. I followed the below steps:

1. Go to File -> New FlexAgent Regex File

2. Load log file

3. Then I generated Regex for each line

4. And at the end I saved the file.

But it didn't worked in my case. Would you mind assisting me to learn this tool?

Thanks,

Aniket


0 Likes

Hi Chris,

Would you please guide me with Regex Utility?

0 Likes
Absent Member.
Absent Member.

regex=\...\s(\d+) (\S+\s+\d+ \d\d:\d\d:\d\d),(\d+) \| (\w+) \W (\w+) \W+([^-]*)\ Web-Worker-Thread : (\d+).*

0 Likes
Fleet Admiral
Fleet Admiral

Not sure if anyone has seen this before - Creating Flex Connectors to use within the Syslog Smart Connector Framework for HP ArcSight - YouTube

But its a great video that runs through the process of creating a FlexConnector using the Regex utility and what goes where. Covers a lot of ground, but its worth going through as there are some fantastic hints and tips there too!

Credit to Shane Lilley for doing it though! Thanks!

0 Likes
Absent Member.
Absent Member.

This is the flex I came up with. You should be able to modify it to suit your needs. As mentioned in this thread your regex is not escaped correctly for the flex connector. You may also want to read the Flex Connector Developer Guide for information regarding to Sub-Messages, Page 104 (Feb 16,2015).

Regular Expression

\[#\| (\d{2} \w{3} \d{4}) (\d{2}:\d{2}:\d{2},\d{3}) \| (\w+) \| (\w+) \| (\w+) \| ([^\(]+)\(([^)]+)\) \| \(\d+\):([^.]+)\. \| ([\w-]+) : (\d{2}).*

Regular Expression in Flex Connector

regex=\\[\#\\| (\\d{2} \\w{3} \\d{4}) (\\d{2}\:\\d{2}\:\\d{2},\\d{3}) \\| (\\w+) \\| (\\w+) \\| (\\w+) \\| ([^\\(]+)\\(([^)]+)\\) \\| \\(\\d+\\)\:([^.]+)\\. \\| ([\\w-]+) \: (\\d{2}).*

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.