Highlighted
Trusted Contributor.. siddarthtalupula1 Trusted Contributor..
Trusted Contributor..
297 views

Customer Mapping

Hello,

Scenario:
The raw events from various Customers are sent via multiple Logstashs to the same ArcSight Connector.

Example:

Customer 1 via Logstash 1 on Port 10120 to ArcSight Smart Connector 1
Customer 2 via Logstash 2 on Port 10121 to ArcSight Smart Connector 1
Customer 3 via Logstash 3 on Port 10122 to ArcSight Smart Connector 1

1. Given the scenario, how can I map the customer on ESM?
2. Can we configure the same connector to listen on multiple ports?


Please advise. The management would like to use the same connector for multiple customers.

 

--
Thanks and Regards
Siddarth T

 

 

0 Likes
10 Replies
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: Customer Mapping

You can do that using an asset and network model in ESM.
The asset and network model will enrich the events with a customer URI which help to identify the logs that belong for customer X or Y or Z ...etc

-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Edouard
Trusted Contributor.. siddarthtalupula1 Trusted Contributor..
Trusted Contributor..

Re: Customer Mapping

Hi@Edouard Pernot1 ,

 

How to deal wih cases when there are multiple customers from the same connector and incases of IP Address overlap between customers?

 

--
Thanks and Regards,
Siddarth T

0 Likes
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: Customer Mapping

wait a sec, are you saying 2 devices from different customer having the same IP sending logs to the same connector ? if yes there might be something wrong. The network overlap is taking care with the fact that you have 1 network and asset model per customer so you should not have any problem with this because the connector knows which network asset model to pick based on the device address.
What kind of connector are we talking about ? are you talking about a chained connector, in that case you might to do some manual work with map files.
0 Likes
Trusted Contributor.. siddarthtalupula1 Trusted Contributor..
Trusted Contributor..

Re: Customer Mapping

Hi@Edouard Pernot1 

I am referring to multiple customers on the same connector with overlapping IP Address. How can I differenciate between 2 customers who come from the same connector and with over lapping addresses?

I have a syslog connector that is receiving feed from multiple customers.

--
Thanks and Regards,
Siddarth T

0 Likes
Super Contributor.. kUMters Super Contributor..
Super Contributor..

Re: Customer Mapping

Hi, there is no way to do it on one connector without prefiltering/tagging...

There is two ways:

- use two connectors with different NETWORK/CUSTOMER URI.

- use different ports on syslog-ng for incoming logs, one port per customer. Then tag this logs in syslog-ng and forward them to a connector. In the connector, properties reconfigure header parsing to parse the CUSTOMER  tag... This way we have been using if we had more customers in one connector. It's working solution, but complicated and we rather switched to connectors deployed in the customer environment - Every customer has own Collector Server with more connectors...

Solution Security Architect
Trusted Contributor.. siddarthtalupula1 Trusted Contributor..
Trusted Contributor..

Re: Customer Mapping

Hi @kUMters ,

 

Can you please elaborate the second option (use different ports on syslog-ng for incoming logs, one port per customer).

I feel this approach is more near to the customer requirement.

 

--
Thanks and Regards,
Siddarth T

0 Likes
Super Contributor.. kUMters Super Contributor..
Super Contributor..

Re: Customer Mapping

Hi Siddarth,

I get over our Knowledge Base and this is what I found:

  • in syslog-ng, we have had rewrites for every customer which will rewrite syslog HEADER and insert "Customer=XXX". So rewritten HOSTNAME looks: "Customer=XXX|$hostname" (example: "Customer=AXENTA|exchange.axenta.local")
  • In SC you have to overwrite syslog header parsing (I'm not able to find where). This is what I found in our KB:

 

syslog.header.hostname=(?s)^(?\:Customer\=[^\\|]*\\||)(?\:([^ \:\\[\\]\=]*[a-zA-Z][^ \:\\[\\]\=]*)\\s+)?(.*)
syslog.header.timestamp.ip=(?s)^(?\:Customer\=[^\\|]*\\||)([A-Z][a-z]{2}\\s+\\d+\\s+\\d{1,2}\:\\d{2}\:\\d{2})\\s+(?\:\\[(\\d+\\.\\d+\\.\\d+\\.\\d+)\\.\\d+\\.\\d+\\]\\s+)?(?\:(\\d+\\.\\d+\\.\\d+\\.\\d+)\\s+)?(?\:([a-fA-F0-9\:\\.]+\\\:[a-fA-F0-9\:\\.]+)\\s+)?(.*)
syslogng.header=(?s)^(?\:Customer\=[^\\|]*\\||)(?\:\\d{1})?\\s+(\\S+)\\s+(\\S+)\\s+(.*)
syslogng.header.tag=(?s)^(?\:Customer\=[^\\|]*\\||)(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(-|(?\:\\[\\S+@[^\\]]+\\])+)\\s+(.*)
syslogng.header.timestamp=(?s)^(?\:Customer\=[^\\|]*\\||)(\\d{4}-\\d{2}-\\d{2}T\\d{2}\:\\d{2}\:\\d{2})(\\.\\d+)?(Z|(?\:-|\\+)\\d{2}\:\\d{2})?

 

And mapping file to parse that customer name to CustomerURI:

 

cat map.3.properties
set.expr(rawEvent).event.customerURI
"__regexToken(rawEvent,""^Customer=([^\|]*)\|.*"")"

 

Tomorrow I can ask my colleagues to get more details of this. (We don't use it anymore, but I think we still have it deployed somewhere...)

Solution Security Architect
Super Contributor.. kUMters Super Contributor..
Super Contributor..

Re: Customer Mapping

I found it. Here it is (but as I said it's not the best way how to user SCs):

Syslog-NG rewrite HEADER:

s.gif

ArcSight SC:

#vim /opt/arcsight/agents/container1/current/user/agent/agent.properties

sc.gif

Mapping file for SC to extract CustomerURI:

# cat map.3.properties
	set.expr(rawEvent).event.customerURI
"__regexToken(rawEvent,""^Customer=([^\|]*)\|.*"")"

 

 

Solution Security Architect
ianfitz Outstanding Contributor.
Outstanding Contributor.

Re: Customer Mapping

Yikes ,have fun with that syslog header stuff.

It's way more straightforward, simple, and sustainable to use a connector per customer (ie per Log Stash instance you have). That's how things are *meant* to work. You can hack it many other ways, but all of those ways will have issues (even if the only issue is complexity, which can be an issue in itself).

Surely you can support more than one connector on your host?  

Whichever option you choose, good luck, but please let common sense prevail. Once you're moving onto a new project, the next poor person will have no idea what weird sorcery was used if you try to make it work on one connector.

Cheers!

Super Contributor.. kUMters Super Contributor..
Super Contributor..

Re: Customer Mapping

Yeah, totally agree with you. The solution I described is one of them and from a whole perspective, it's a road to hell. After one year we were able to convince the SOC guys to deploy customer connectors in their networks with dedicated SCs:)

Solution Security Architect
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.