RegEx FlexConnector Not Parsing.

Below I have posted a small set of sample logs, the code for my flexagent parser, and the entry I have in the agents.properties file for a custom syslog parser.  Is there anything that stands out as to why the data is not being parsed?  If I look in the agents.log file, I see the following error: 

[2024-05-30 15:03:42,809][WARN ][com.arcsight.agent.parsers.operation.regexTokenOperation] [getResult]No match between string [May 30 12] and regex [(\S+) \S+ (?:login|sshd|httpd)]

  • Hi  , I'm not sure if the sample logs or the flexagent parser got attached here.  

  • This is the sample of the logs and the Flexagent parser below

    # <185>May 23 2024 11:51:15 %FTD-1-430003: EventPriority: Low, DeviceUUID: dd4d1b20-e37a-11ee-a052-870306bb8e79, InstanceID: 1, FirstPacketSecond: 2024-05-23T11:51:15Z, ConnectionID: 24226, AccessControlRuleAction: Allow, SrcIP: 172.16.21.5, DstIP: 162.232.201.14, SrcPort: 63451, DstPort: 443, Protocol: tcp, IngressInterface: Inside, EgressInterface: Outside, IngressZone: INSIDE, EgressZone: OUTSIDE, IngressVRF: Global, EgressVRF: Global, ACPolicy: AI-PDP-1150-DC-01 Access policy, AccessControlRuleName: New-Rule-#2-ALLOW, Prefilter Policy: Default Prefilter Policy, ConnectionDuration: 0, InitiatorPackets: 1, ResponderPackets: 1, InitiatorBytes: 59, ResponderBytes: 70, NAPPolicy: Balanced Security and Connectivity


    regex=.*<\\d+>([\\w\\d\\s:]+)\\s\\s\\s.*AccessControlRuleAction:\\s([^,]+),\\sSrcIP:\\s([^,]+),\\sDstIP:\\s([^,]+),\\sSrcPort:\\s([^,]+),\\sDstPort:\\s([^,]+),\\sProtocol:\\s([^,]+),\\sIngressInterface:\\s([^,]+),\\sEgressInterface:\\s([^,]+),\\s.*ACPolicy:\\s([^,]+),\\sAccessControlRuleName:\\s([^,]+),\\s.*InitiatorBytes:\\s([^,]+),\\sResponderBytes:\\s([^,]+),.*


    token.count=13


    token[0].name=timestampp
    token[0].type=String

    token[1].name=deviceactions
    token[1].type=String

    token[2].name=sourceaddresses
    token[2].type=IPAddress

    token[3].name=destinationaddresss
    token[3].type=IPAddress

    token[4].name=sourceports
    token[4].type=Integer

    token[5].name=destinationports
    token[5].type=Integer

    token[6].name=protocolss
    token[6].type=String

    token[7].name=deviceoutbounddirections
    token[7].type=String

    token[8].name=deviceinbounddirections
    token[8].type=String

    token[9].name=Policy
    token[9].type=String

    token[10].name=Rule
    token[10].type=String

    token[11].name=bytesinn
    token[11].type=Integer

    token[12].name=bytesoutt
    token[12].type=Integer

    additionaldata.enabled=true

    event.name=__stringConstant("Firewall Events")
    event.timestamp=timestampp
    event.deviceAction=deviceactions
    event.sourceAddress=sourceaddresses
    event.destinationAddress=destinationaddresss
    event.sourcePort=sourceports
    event.destinationPort=destinationports
    event.transportProtocol=protocolss
    event.deviceInboundInterface=deviceinbounddirections
    event.deviceOutboundInterface=deviceoutbounddirections
    event.deviceCustomString1=Policy
    event.deviceCustomString2=Rule
    event.bytesIn=bytesinn
    event.bytesOut=bytesoutt



  • This is stil the agent.properties file below

    #



    regex=(.*)
    token[0].count=1

    token[0].name=initiallogs
    token[0].type=String


    event.rawEvent=initiallogs
    event.deviceVendor=__StringContant("CISCO")
    event.deviceProduct=__StringContant("FTD")


    additionaldata.enabled=true

    extraprocessor.count=1

    extraprocessor[0].type=regex
    extraprocessor[0].filename=ftdparser/log_ftd
    extraprocessor[0].field=event.rawEvent
    extraprocessor[0].flexagent=true
    extraprocessor[0].clearfieldafterparsing=false

  • Assuming these events are coming over syslog.  Give this a try:

    # FlexAgent Regex Configuration File
    do.unparsed.events=true

    regex=([\\w\\d\\s\:]+)\\s[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,\\s+AccessControlRuleAction\:\\s([^,]+),\\sSrcIP\:\\s([^,]+),\\sDstIP\:\\s([^,]+),\\sSrcPort\:\\s([^,]+),\\sDstPort\:\\s([^,]+),\\sProtocol\:\\s([^,]+),\\sIngressInterface\:\\s([^,]+),\\sEgressInterface\:\\s([^,]+),\\s.*ACPolicy\:\\s([^,]+),\\sAccessControlRuleName\:\\s([^,]+),\\s.*InitiatorBytes\:\\s([^,]+),\\sResponderBytes\:\\s([^,]+),.*

    token.count=13

    token[0].name=timestampp
    token[0].type=TimeStamp
    token[0].format=MMM dd yyyy HH\:mm\:ss

    token[1].name=deviceactions
    token[1].type=String

    token[2].name=sourceaddresses
    token[2].type=IPAddress

    token[3].name=destinationaddresss
    token[3].type=IPAddress

    token[4].name=sourceports
    token[4].type=Integer

    token[5].name=destinationports
    token[5].type=Integer

    token[6].name=protocolss
    token[6].type=String

    token[7].name=deviceoutbounddirections
    token[7].type=String

    token[8].name=deviceinbounddirections
    token[8].type=String

    token[9].name=Policy
    token[9].type=String

    token[10].name=Rule
    token[10].type=String

    token[11].name=bytesinn
    token[11].type=Integer

    token[12].name=bytesoutt
    token[12].type=Integer


    #submessage.messageid.token=
    #submessage.token=

    additionaldata.enabled=true


    event.sourcePort=sourceports
    event.deviceAction=deviceactions
    event.deviceCustomString2=Rule
    event.deviceCustomString1=Policy
    event.name=__stringConstant("Firewall Events")
    event.destinationPort=destinationports
    event.sourceAddress=sourceaddresses
    event.destinationAddress=destinationaddresss
    event.transportProtocol=protocolss
    event.bytesIn=bytesinn
    event.bytesOut=bytesoutt
    event.deviceInboundInterface=deviceinbounddirections
    event.deviceReceiptTime=timestampp
    event.deviceOutboundInterface=deviceoutbounddirections


    #l10n.filename.prefix=