Flexconnector Parser question

Hello,

I am trying to write a custom parser for a couple of events we are getting from a PBX into a syslog connector. The generic syslog parser handles the most of the events, but there are a couple that it is not handling correctly which I am trying to parse.

I have create the following 2 parser files:

NortelPBXLoginSucess.sdkrfilereader.properties &

NortelPBXLogout.sdkrfilereader.properties

and placed them in:

/home/ArcSightSmartConnectors/syslogconnector/current/user/agent/flexagent/syslog

and changed their file permissions to 777.

The first looks for logout events:

Dec  9 10:19:05 foo INFO: : 10.95.3.201: Info: User goo logout from server foo.boo.com:443.
Dec  9 10:20:55 foo INFO: : 10.95.3.201: Info: User goo logout from server foo.boo.com:443.
Dec  9 10:23:41 foo INFO: : 10.95.3.201: Info: User goo logout from server foo.boo.com:443.
Dec  9 10:25:25 foo INFO: goo: 10.95.3.201: Info: User goo logout from foo.boo.com:443.
Dec  9 10:25:38 foo INFO: goo: 10.95.3.201: Info: User goo logout from foo.boo.com:443.
Dec  9 10:25:49 foo INFO: goo: 10.95.3.201: Info: User goo logout from foo.boo.com:443.
Dec  9 10:26:32 foo INFO: goo: 10.95.3.201: Info: User goo logout from server foo.boo.com:443.
Dec  9 12:32:56 foo INFO: goo: 10.95.3.201: Info: User goo logout from server foo.boo.com:443.

The parser file for the logout event is as follows:

# FlexConnector Regex Configuration File
do.unparsed.events=false

regex=(\\w{3}\\s \\d{1,2}\\s \\d{2}:\\d{2}:\\d{2})\\s (\\w )\\s INFO:\\s*(\\w*):\\s (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}):\\s Info:\\s (.*):(\\d )

token.count=6

token[0].name=Timestamp
token[0].type=TimeStamp
token[0].format=MMM  dd HH\:mm\:ss
token[1].name=HostName
token[1].type=String
token[2].name=UserName
token[2].type=String
token[3].name=DestinationAddress
token[3].type=IPAddress
token[4].name=Message
token[4].type=String
token[5].name=DestinationPort
token[5].type=Integer

event.deviceReceiptTime=__useCurrentYear(Timestamp)
event.destinationAddress=DestinationAddress
event.destinationPort=DestinationPort
event.destinationHostName=HostName
event.destinationUserName=UserName
event.destinationUserId=UserName
event.name=__stringConstant("Logout")
event.message=Message
event.deviceVendor=__stringConstant("Nortel")
event.deviceProduct=__stringConstant("PBX")

I stopped the connector, deleted the syslog.properties file, and restarted the service but it is still not parsing.

Idaes?

Thanks!

  • The 2nd parser is a multi line file used to aggregate two events in order to extract enough info for it to be worth wile.

    Events:

    Dec  8 17:11:23 boo INFO: : 10.95.3.201: Info: id=goo
    Dec  8 17:11:23 boo INFO: : 10.95.3.201: Info: Login Success

    parser:

    # FlexConnector Regex Configuration File
    do.unparsed.events=false

    multiline.starts.regex=(\\w{3}\\s \\d{1,2}\\s \\d{2}:\\d{2}:\\d{2})\\s (\\w )\\s INFO:\\s :\\s (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}):\\s Info:\\s id=(\\w )
    multiline.ends.regex=(Info:\\sLogin\\sSuccess)
    regex=(\\w{3}\\s \\d{1,2}\\s \\d{2}:\\d{2}:\\d{2})\\s (\\w )\\s INFO:\\s :\\s (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}):\\s Info:\\s id=(\\w )

    token.count=4

    token[0].name=Timestamp
    token[0].type=TimeStamp
    token[0].format=MMM  dd HH\:mm\:ss
    token[1].name=HostName
    token[1].type=String
    token[2].name=DestinationAddress
    token[2].type=IPAddress
    token[3].name=UserName
    token[3].type=String

    event.deviceReceiptTime=__useCurrentYear(Timestamp)
    event.destinationAddress=DestinationAddress
    event.destinationHostName=HostName
    event.destinationUserName=UserName
    event.destinationUserId=UserName
    event.name=__stringConstant("Authen OK")
    event.message=__stringConstant("Login Sucess")
    event.deviceVendor=__stringConstant("Nortel")
    event.deviceProduct=__stringConstant("PBX")

    thanks!

  • So I have been fighting with this and have made no real progress. The current version of my logout parser looks like:

    # FlexAgent Regex Configuration File
    do.unparsed.events=false

    regex=.*INFO\:\\s (\\S )\:\\s (\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})\:\\s Info\:\\s(.*)server\\s(.*)\:(\\d ).

    token.count=5

    token[0].name=UserName
    token[0].type=String

    token[1].name=DestinationAddress
    token[1].type=IPAddress

    token[2].name=Message
    token[2].type=String

    token[3].name=HostName
    token[3].type=String

    token[4].name=DestinationPort
    token[4].type=Integer

    #submessage.messageid.token=

    #submessage.token=

    event.destinationPort=DestinationPort
    event.deviceVendor=__stringConstant("Nortel")
    event.name=__stringConstant("Logout")
    event.destinationUserName=UserName
    event.destinationAddress=DestinationAddress
    event.message=__concatenate(Message,"server ", HostName, ":", DestinatoinPort)
    event.destinationHostName=HostName
    event.deviceProduct=__stringConstant("PBX")
    event.destinationUserId=UserName
    event.deviceSeverity=_SYSLOG_PRIORITY
    event.sourceAddress=_SYSLOG_SENDER
    event.deviceReceiptTime=__useCurrentYear(_SYSLOG_TIMESTAMP)

    Works fine in the Regex tester on the following evetns:

    goo.us.com INFO: boo: 10.5.1.187: Info: User boo logout from server goo.us.com:443.

    or

    DEC 14 17:32:35 goo.us.com INFO: boo: 10.5.1.187: Info: User boo logout from servergoo.us.com:443.

    event log looks like:

    [2010-12-14 17:26:20,679][INFO ][default.com.arcsight.agent.p.e][getInputStream] Resource [syslog/NortelPBXLogout.sdkrfilereader.properties] not found
    [2010-12-14 17:26:20,679][INFO ][default.com.arcsight.agent.p.e][getInputStream] Resource [syslog/NortelPBXLogout.sdkrfilereader.properties] not found (AUP file ignored)
    [2010-12-14 17:26:20,679][INFO ][default.com.arcsight.common.config.AgentPropertiesFileConfiguration][customInitialization] customInitialization() - read properties from file [/apps/arcsight/ArcSightSmartConnectors/syslogconnector/current/user/agent/flexagent/syslog/NortelPBXLogout.sdkrfilereader.properties].
    [2010-12-14 17:26:20,680][INFO ][default.com.arcsight.agent.sdk.d.n][init] Successfully Parsed properties from file [syslog/NortelPBXLogout]

    Any advice is appreciated.

  • Well, I figured out where my events are going. They are parsing as Cisco IronPort evetns. I tried changing the precedence in the syslog.properties file, but the connector seems to ignore my changes and then overwrites them restoring the old order.

    Ideas?