Highlighted
Arcsight_Logger_User Super Contributor.
Super Contributor.
860 views

Flexconnector for parsing HTTP headers

Jump to solution

Hi,

We are currently trying to parse Airwatch SEG Listener logs but could not find a suitable flexconnector to use. (no native Airwatch connectors available)

The log source is in a .log file format and the format of the logs is not 1 event per line: (amber is the tab delimited event and blue is the associated header)

Which flexconnector type would be the best to parse this?

Thanks!
Kevin

2018/10/30 08:27:21.750 XXXXXXXX013 80000610-0001-9c00-b63f-84710c7967bb [0000000-0000000] (101) Debug
AW.Eas.Web.Listener.ProxyGateway.ModifyResponseHeaders HTTP Response SENT for id 870d7ad4-d65e-4cd2-9609-f312ad18e92b:
HTTP/?.? 200(OK) OK
Content-Encoding: gzip
request-id: 53ec8d0c-e00c-49b7-95ed-ed4afade9cf7
X-CalculatedBETarget: XXXXXXXXXXX035.mail.net.x.y
X-MS-BackOffDuration: L/-469
X-DiagInfo: XXXXXXXXXXX035
X-BEServer: XXXXXXXXXXX035
Persistent-Auth: false
X-FEServer: XXXXXXXXXXX003
Content-Length: 897
Content-Type: application/vnd.ms-sync.wbxml
Date: Tue, 30 Oct 2018 08:27:21 GMT
Set-Cookie: X-BackEndCookie=S-1-5-21-1216582894-834684500-1334827815-153234=u56Lnp2ejJqBnM6cycfGys/Sx5nKy9LLms/P0p2cmczSys/Lz5zKx87OncfNgYHNz87H0s7O0s3Gq8/Hxc3Ixc3O;
expires=Thu, 29-Nov-2018 08:27:21 GMT; path=/Microsoft-Server-ActiveSync; secure; HttpOnly; Domain=domain.local
WWW-Authenticate: Negotiate YIGZBgkqhkiG9xIBAgICAG+BiTCBhqADAgEFoQMCAQ+iejB4oAMCARKicQRvfWf/zv2j0yyp0kwfYRloENqVJ6Qa5615F/BRz55efWSW7W6wVI3HZFlNVu
+PbMOgUECCBehW2gDoKxZjXYiH4Ss1hlQ9NGOBVrEjBBPvN2FH9p9ypJxpS1yacxuIYKcvjXx+sfq+PIcJL2oQ4z0L
Cache-Control: private
X-AspNet-Version: 4.0.30319

Labels (1)
Tags (1)
0 Likes
1 Solution

Accepted Solutions
Micro Focus Expert
Micro Focus Expert

Re: Flexconnector for parsing HTTP headers

Jump to solution

The correct parser to use here would be a regex multi-line parser.

Documentation about that can be found here: https://community.softwaregrp.com/t5/ArcSight-Connectors/HPE-ArcSight-FlexConnector-Developer-s-Guide/ta-p/1584874?nm

The specific you are looking for starts on page 127, including multi-line examples.

In most cases you set a start regex and a end regex, and it will match all lines in between, hopefully the examples are sufficient, and if not feel free to ask away here on the community page if you get into any issues!

-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
5 Replies
Micro Focus Expert
Micro Focus Expert

Re: Flexconnector for parsing HTTP headers

Jump to solution

The correct parser to use here would be a regex multi-line parser.

Documentation about that can be found here: https://community.softwaregrp.com/t5/ArcSight-Connectors/HPE-ArcSight-FlexConnector-Developer-s-Guide/ta-p/1584874?nm

The specific you are looking for starts on page 127, including multi-line examples.

In most cases you set a start regex and a end regex, and it will match all lines in between, hopefully the examples are sufficient, and if not feel free to ask away here on the community page if you get into any issues!

-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
Arcsight_Logger_User Super Contributor.
Super Contributor.

Re: Flexconnector for parsing HTTP headers

Jump to solution

Thanks for the kind help~ it is the exact solution! I didn't make it past chapter 3
Will try it out and update

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: Flexconnector for parsing HTTP headers

Jump to solution

Good to hear @Arcsight_Logger_User ! If you manage to figure it out, it would be great if you could accept an answer as a solution.

-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
0 Likes
Arcsight_Logger_User Super Contributor.
Super Contributor.

Re: Flexconnector for parsing HTTP headers

Jump to solution

Hi Marius,

Thanks for the advice~ It worked and I am seeing the multi-line logs appended as space separated values.

Took a bit to figure out the arcsight regex tool and replace the auto-generated regex expressions to cover what I wanted (Airwatch mixes space and tab in their logs).

Thanks again!

Regards,
Kevin

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: Flexconnector for parsing HTTP headers

Jump to solution

Good to hear!

Yeah i never really use that regex tool, i find the regex created either too specific or too broad, it does have good usercases though.

Normally i would just use a online or offline tool like https://regex101.com/ , then when i finalize the regex, i would use the regex tool to parse all the logfiles, to ensure that my regex hits all content i need, but we all have different ways of working 🙂

-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.