Highlighted
etha4u Trusted Contributor.
Trusted Contributor.
726 views

Flex connector Categorization file is not reading by the syslog subparser connector

I have keep the file below path and it has above heading

the categorization file nips.csv in ARCSIGHT_HOME/user/agent/acp/categorizer/current/im/

the categorization file input attached, but it's not reading; any input will be a great help

Thanks

updated files as requested/suggested

Message was edited by: Fahima Khan

0 Likes
10 Replies
Respected Contributor.. george_m_c Respected Contributor..
Respected Contributor..

Re: Flex connector Categorization file is not reading by the syslog subparser connector

Hi Fabima,

To understand more about your current issue, you may need to share exact directory structure used for custom categorization. Given path mentioned in your post is not right.

<device_vendor>/<device_product>.csv

eg: ibm/nips.csv

If this is not solving your issue then you may need to use map type extra.processor in your custom sub-agent parser.

extraprocessor.count=1

extraprocessor[0].type=map

extraprocessor[0].filename=ibm/nips.csv

Regards,

BL

0 Likes
etha4u Trusted Contributor.
Trusted Contributor.

Re: Flex connector Categorization file is not reading by the syslog subparser connector

the path is as below

Let me post the header of the file as below:

        

event.deviceEventClassIdset.event.categoryBehaviorset.event.categoryDeviceGroupset.event.categoryObjectset.event.categorySignificanceset.event.categoryTechniqueset.event.categoryOutcomeset.event.deviceAction
Ping_Sweep/Communicate/Query/IDS/Network/Network/Recon/Scan/AttemptDetected event
TCP_443_Protocol_Unknown/Communicate/Query/IDS/Network/Host/Application/Service/Suspicious/Traffic Anomaly/AttemptAttack failure (blocked by Proventia appliance)
Echo_Reply_Without_Request/Communicate/IDS/Network/Host/Compromise/DoS/AttemptDetected event
SIP_Contact_From_Id_Mismatch/Communicate/Query/IDS/Network/Host/Application/Suspicious/Traffic Anomaly/Application Layer/AttemptDetected event
MSRPC_Invalid_Request/Communicate/Query/IDS/Network/Host/Application/Service/Suspicious/Traffic Anomaly/Application Layer/AttemptDetected event
Trace_Route_UDP/Communicate/Query/IDS/Network/Host/Recon/Exploit/Weak Configuration/AttemptDetected attack (vuln not scanned recently)
TCP_443_Protocol_Unknown/Communicate/Query/IDS/Network/Host/Application/Service/Suspicious/Traffic Anomaly/AttemptSimulated block (blocking not enabled)

as per my parser below is the value of product vendor and

event.categoryDeviceGroup=__stringConstant("/IDS/Network")

event.deviceVendor=__stringConstant("IBM")

event.deviceCustomString1=Message

event.deviceProduct=__stringConstant("NIPS")

so i am using the path ibm

file name is nips.csv

I will try the extra mapping in my code and will update you shortly.

Thanks a million for your suggestion.

0 Likes
etha4u Trusted Contributor.
Trusted Contributor.

Re: Flex connector Categorization file is not reading by the syslog subparser connector

do I need to change anything in agent.properties file?

I am getting below in flex dev guide:

0 Likes
jorgeoa Honored Contributor.
Honored Contributor.

Re: Flex connector Categorization file is not reading by the syslog subparser connector

It sounds obvious but, are you sure that the deviceEventClassId of the generated events matches the ones in the categorization file?

0 Likes
etha4u Trusted Contributor.
Trusted Contributor.

Re: Flex connector Categorization file is not reading by the syslog subparser connector

from initial look it seems same

0 Likes
jorgeoa Honored Contributor.
Honored Contributor.

Re: Flex connector Categorization file is not reading by the syslog subparser connector

Could you attach the last version of your categorization file and the agent.log file when the first event arrives? After start the connector, when the first event of a device vendor arrives, it loads the categorization files so we can see if it is loaded correctly.

0 Likes
etha4u Trusted Contributor.
Trusted Contributor.

Re: Flex connector Categorization file is not reading by the syslog subparser connector

attached are here, after using extra mapping the properties file is not even parsing.

1. Properties file implemented:

(See attached file: nipsjune.subagent.sdkrfilereader.properties)

2. location of .csv file

3. Agent.propertes:

(See attached file: agent.log)

0 Likes
etha4u Trusted Contributor.
Trusted Contributor.

Re: Flex connector Categorization file is not reading by the syslog subparser connector

After I incorporate this change it is not parsing the log anymore. Do I need to change anything in agent.properties file?

0 Likes
balahasan.v1 Acclaimed Contributor.
Acclaimed Contributor.

Re: Flex connector Categorization file is not reading by the syslog subparser connector

Hey Fahima,

If u parser is working fine. then plz do the below changes

> Plz remove the set.event.deviceAction from the categorizer file and please do the deviceaction mapping in a map.x.properties file.

> Don't mention full path in the extraprocessor

extraprocessor.count=1

extraprocessor[0].type=map

extraprocessor[0].filename=ibm/nips.csv

extraprocessor[0].allowoverwrite=false

extraprocessor[0].casesensitive=false

extraprocessor[0].charencoding=US-ASCII

extraprocessor[0].trimgettertokens=false

extraprocessor[0].trimsettertokens=false

0 Likes
Established Member.. alexs@obcrest.c1
Established Member..

Re: Flex connector Categorization file is not reading by the syslog subparser connector

Not sure if you have fixed the problem. Anyway, here is what I found could fix your problem.

In your parser properties file, you have the following

>>event.categoryDeviceGroup=__stringConstant("/IDS/Network")

Please remove this entry. I found that if you have entries for any of the event.category fields, the connector would not pick up the categorizer file.  Once this entry is removed, the connector should use the categorizer file. You can set the deviceGroup in your categorizer file anyway.

Also, in your parser, if you use extraprocessor map, then the map file should be placed under  \user\agent\fcp.

Here is the extract from the FlexConnector development guide, under the extraprocessor section

---------------------

Except for the map extra processor configuration file, all extra processor configuration files

should be placed in the \user\agent\flexagent folder. The map extra processor file

should be placed in \user\agent\fcp or \user\agent\aup\fcp. If a map

configuration file exists in both the paths, the one in \user\agent\aup\fcp overrides

the one in \user\agent\fcp.

---------------------

However, if you are just mapping categories, then the categorizer is a better place to do. extraprocessor mapping is best for mapping other fields.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.