Highlighted
Trusted Contributor.. bireland1 Trusted Contributor..
Trusted Contributor..
1064 views

Trouble with Syslog Flexconnector

Jump to solution

I am currently attempting to apply a flex connector to parse the current syslog data received from a pair of Cisco Content engines. there is a screenshot of what I currently receive, the agent.properties file and a copy of the properties file I would like to apply. Any help on what I'm missing would be greatly appreciated.

Labels (2)
Tags (3)
0 Likes
1 Solution

Accepted Solutions
eugene.rostofsk1
New Member.

Re: Trouble with Syslog Flexconnector

Jump to solution

Hi Brian,

You should not have tried to parse the following portion of your syslog messages: "<46>Sep 26 22:21:43 rpcysd-godc1-ce02" - Syslog Connector automatically parses it by itself:

Tokens Available for Syslog Parsers Only

Token StringDescription
_SYSLOG_TIMESTAMPTime stamp received in the header of the syslog message
_SYSLOG_SENDERHost name or IP address of the sender received in the header of the syslog message
_SYSLOG_FACILITYFacility received in the header of the syslog message (applies only to Syslog Daemon connector)
_SYSLOG_PRIORITYPriority received in the header of the syslog message (applies only to Syslog Daemon connector)

Try parsing starting from "cache:" or even " %CE-TRNSLG-6-460012:". I tested the following regex (it is far from perfect - more of a quick fix for demonstration) on your sample messages:

(.*)(%.*):\\s+(\\d+.\\d+)\\s+(\\d+)\\s+(\\d{1,3}.\\d{1,3}.\\d{1,3}.\\d{1,3})\\s+([\\w/]+)\\s+(\\d+)\\s+(\\w+)\\s+(http[^\\s]+)\\s+\\-\\s+([^\\s]+)\\s+\\-\\s+(\\w+)\\s+""(\\w+)""

Make sure you have your websense.subagent.sdkrfilereader,properties in the user/agent/flexagent/syslog/ directory, usecustomsubagentlist set to true and that flexagent_syslog tops the customsubagentlist property in the agent.properties file.

HTH

Igor

View solution in original post

0 Likes
7 Replies
cgi1 Absent Member.
Absent Member.

Re: Trouble with Syslog Flexconnector

Jump to solution

Hi Brian,

you should set in agent.properties as minimum

agents[0].usecustomsubagentlist=true

Thus the connector reads you definition in

agents[0].customsubagentlist=flexagent_syslog|generic_syslog

For Flexagents it is always flexagent_syslog there not your subagent name.

Regards

Christian

0 Likes
balahasan.v1 Acclaimed Contributor.
Acclaimed Contributor.

Re: Trouble with Syslog Flexconnector

Jump to solution

Hi Brian,

Have u checked the parser (\\S+\\s+\\d+ \\d\\d\:\\d\\d\:\\d\\d) rpcysd\\-(\\S+) cache\: %CE\\-TRNSLG\\-)

Don't give blank space in the regex. Please replace them with (\\s+)

Ex:

(\\S+\\s+\\d+ \\d\\d\:\\d\\d\:\\d\\d) rpcysd\\- --> (\\S+\\s+\\d+ \\d\\d\:\\d\\d\:\\d\\d)\\s+rpcysd\\-

0 Likes
cloudwang Regular Contributor.
Regular Contributor.

Re: Trouble with Syslog Flexconnector

Jump to solution

Would you mind show us some syslog sample of your case?

It help find out the exactly problem.

0 Likes
Trusted Contributor.. bireland1 Trusted Contributor..
Trusted Contributor..

Re: Trouble with Syslog Flexconnector

Jump to solution

Sorry for the delay yes please see attached:

"<46>Sep 26 22:21:43 rpcysd-godc1-ce02 cache: %CE-TRNSLG-6-460012: 1380234103.006 176 10.00.00.00 TCP_MISS/200 760 POST http://grooveshark.com/more.php?markSongDownloadedEx - DIRECT/grooveshark.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:21:43 rpcysd-godc1-ce02 cache: %CE-TRNSLG-6-460012: 1380234103.149 256 10.00.00.00 TCP_MISS/200 378 GET http://www.snl.com/SNLWebPlatform/Services/Ajax/News/NewsWire.svc/PingSession - DIRECT/www.snl.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:44 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235304.250 112 10.00.00.00 TCP_MISS/200 511 GET http://turnerhd-f.akamaihd.net/z/tvecnn_1@135347/tiny_23076c82f663a305-p.bootstrap?g=URCFCSLISVDB&hdcore=3.1.0&plugin=aasp-3.1.0.43.124 - DIRECT/turnerhd-f.akamaihd.net - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:21:43 rpcysd-godc1-ce02 cache: %CE-TRNSLG-6-460012: 1380234103.182 240 10.00.00.00 TCP_MISS/200 1873 GET http://www.snl.com/SNLWebPlatform/Services/Ajax/News/NewsWire.svc/GetItemSet?since=%22%5C%2FDate(1380215264000)%5C%2F%22&lastKey=%22%22&setSize=100 - DIRECT/www.snl.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:15:15 nebv-codc1-ip02 cache: %CE-TRNSLG-6-460012: 1380233715.919 80 10.00.00.00 TCP_MISS/200 399 GET http://www.google-analytics.com/__utm.gif?utmwv=4.3as&utmn=154212369&utmhn=www.huskers.com&utmt=event&utme=5(Video%20Live*Video%20Duration*981287:Sports%20Nightly)(15)&utmcs=UTF-8&utmsr=1440x900&utmsc=32-bit&utmul=en-us&utmje=0&utmfl=11.7%20r255&utmdt=Sports%20Nightly%20-%20Huskers.com%20-%20Nebraska%20Athletics%20Official%20Web%20Site&utmhid=1832076396&utmr=-&utmp=/mediaPortal/player.dbml?&id=981... - DIRECT/www.google-analytics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:44 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235304.666 288 10.00.00.00 TCP_MISS/200 115222 GET http://turnerhd-f.akamaihd.net/z/tvecnn_1@135347/tiny_23076c82f663a305-p_Seg1-Frag230039802?als=7.51,12,30.03,0,149,2074,30,158,0,317,f,3531665.03,3531678,t,s,URCFCSLISVDB,3.1.0,317&hdcore=3.1.0&plugin=aasp-3.1.0.43.124 - DIRECT/turnerhd-f.akamaihd.net - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:44 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235304.714 128 10.00.00.00 TCP_MISS/302 530 POST http://go.microsoft.com/fwlink/?LinkID=88343 - DIRECT/go.microsoft.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:21:43 rpcysd-godc1-ce02 cache: %CE-TRNSLG-6-460012: 1380234103.726 20849 10.00.00.00 TCP_MISS/200 611 GET http://stream1.sportsillustrated.fyre.co/v3.0/collection/47453162/1379560322683623/?jid&siteId=321264&networkId=sportsillustrated.fyre.co&backend=DTH - DIRECT/stream1.sportsillustrated.fyre.co - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:44 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235304.826 80 10.45.10.204 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/blank.gif - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:21:43 rpcysd-godc1-ce02 cache: %CE-TRNSLG-6-460012: 1380234103.742 112 10.2.10.160 TCP_MISS/200 1188 GET http://stream1.foxnewsprod.fyre.co/v3.0/collection/48334186/1380238657472015/?jid&siteId=310256&networkId=foxnewsprod.fyre.co&backend=ELB&callback=_callbacks_._2khm2mk6r2 - DIRECT/stream1.foxnewsprod.fyre.co - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:44 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235304.842 79 10.45.10.204 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/topBarBg.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:44 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235304.874 80 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/searchBar.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:44 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235304.874 80 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/btnBg.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:44 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235304.906 160 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/topBg2.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:21:43 rpcysd-godc1-ce02 cache: %CE-TRNSLG-6-460012: 1380234103.822 544 10.00.00.00 TCP_MISS/200 274 GET http://rm.api.weibo.com/2/remind/push_count.json?trim_null=1&exclude_attitude=1&msgbox=true&_pid=10001&count=691&source=3818214747&status_type=0&with_closefriends=true&callback=STK_13802183018561638 - DIRECT/rm.api.weibo.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:44 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235304.922 79 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/popDownBg.png - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:21:43 rpcysd-godc1-ce02 cache: %CE-TRNSLG-6-460012: 1380234103.854 528 10.00.00.00 TCP_MISS/200 351 GET http://rm.api.weibo.com/2/remind/unread_hint.json?source=3818214747&with_url=1&appkeys=2083995643,603152360,99075054,2842762591,3073740076,2698881363,2882603098,3654374917,2936099636,2900908525,0,125932898,872034675,3845272542&group_ids=201110150511927294,201110150518199978,3412068260833968&callback=STK_13802183018561640 - DIRECT/rm.api.weibo.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:45 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235305.018 159 10.00.00.00 TCP_MISS/500 4109 GET http://www.doaneathletics.com/images_web/menuWatermark.png - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:45 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235305.018 96 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/headerImg.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:45 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235305.018 191 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/navBg2.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:45 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235305.034 80 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images/sportHeaders/WomensSoccer2.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:45 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235305.034 80 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images/sportSocMediaIcons/sportFacebookIcon.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:45 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235305.050 79 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images/sportSocMediaIcons/sportTwitterIcon.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:45 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235305.082 32 10.00.00.00 TCP_IMS_HIT/304 151 GET http://p.sidhelp.com/c/images/overlays/trans25.png - NONE/- - ALLOW ""WEBSENSE"""

"<46>Sep 26 22:41:45 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235305.130 79 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/gpacLogo.png - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

0 Likes
Trusted Contributor.. bireland1 Trusted Contributor..
Trusted Contributor..

Re: Trouble with Syslog Flexconnector

Jump to solution

Looking at the syslog.properites file it is updated by the connector and is applying passthrough_syslog to devices not the custom parser.

0 Likes
eugene.rostofsk1
New Member.

Re: Trouble with Syslog Flexconnector

Jump to solution

Hi Brian,

You should not have tried to parse the following portion of your syslog messages: "<46>Sep 26 22:21:43 rpcysd-godc1-ce02" - Syslog Connector automatically parses it by itself:

Tokens Available for Syslog Parsers Only

Token StringDescription
_SYSLOG_TIMESTAMPTime stamp received in the header of the syslog message
_SYSLOG_SENDERHost name or IP address of the sender received in the header of the syslog message
_SYSLOG_FACILITYFacility received in the header of the syslog message (applies only to Syslog Daemon connector)
_SYSLOG_PRIORITYPriority received in the header of the syslog message (applies only to Syslog Daemon connector)

Try parsing starting from "cache:" or even " %CE-TRNSLG-6-460012:". I tested the following regex (it is far from perfect - more of a quick fix for demonstration) on your sample messages:

(.*)(%.*):\\s+(\\d+.\\d+)\\s+(\\d+)\\s+(\\d{1,3}.\\d{1,3}.\\d{1,3}.\\d{1,3})\\s+([\\w/]+)\\s+(\\d+)\\s+(\\w+)\\s+(http[^\\s]+)\\s+\\-\\s+([^\\s]+)\\s+\\-\\s+(\\w+)\\s+""(\\w+)""

Make sure you have your websense.subagent.sdkrfilereader,properties in the user/agent/flexagent/syslog/ directory, usecustomsubagentlist set to true and that flexagent_syslog tops the customsubagentlist property in the agent.properties file.

HTH

Igor

View solution in original post

0 Likes
katzmandu1 Absent Member.
Absent Member.

Re: Trouble with Syslog Flexconnector

Jump to solution

You have a bunch of things going on....

1) The proper filename is "VENDOR.subagent.sdkrfilereader.properties" and it lives in $ARCSIGHT_HOME/user/agent/flexagent/syslog ... you're sending events in via syslog, but treating this as a file-reader flexconnector. That won't work.

2) You probably need to hack on your original regex= statement. The syslog parser takes care of the timestamp and device host for you, so your regex will start with the message after the hostname....

# <46>Sep 26 22:41:45 nebv-codc1-ip01 cache: %CE-TRNSLG-6-460012: 1380235305.018 96 10.00.00.00 TCP_MISS/304 247 GET http://www.doaneathletics.com/images_web/headerImg.jpg - DIRECT/www.doaneathletics.com - ALLOW ""WEBSENSE"""

Regex for that, as a syslog subagent may look like

regex=cache:\\s+%(\\S+)\:\\s+(\\d+\\.\\d+)\\s+(\\d+)\\s+(\\d+\\.\\d+\\.\\d+\\.\\d+)\\s+(\\w+)\\/(\\d+)\\s+(\\d+)\\s+(\\S+)\\s+(\\S+)\\s+\\-\\s+(\\w+)\\/(\\S+)\\s+\\-\\s+(\\S+)\\s+(\\S+)

[ There are some errors there, this is quick-n-dirty for demo purposes. ]

If you have more questions or want more help, please let me know.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.