Having problems with your account or logging in?
A lot of changes are happening in the community right now. Some may affect you. READ MORE HERE
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor
2183 views

Difference between a Smart Connector and Smart Collector

Jump to solution

In ArcMC web interface and in docs, i see reference to Smart Collector. Unfortunately i cannot find an explaination which distinguishes collector with connector. 

Technically, what is the difference between a SmartConnector and a SmartCollector and where do we use collector ?

0 Likes
1 Solution

Accepted Solutions
viktor.doundako Respected Contributor.
Respected Contributor.

Re: Difference between a Smart Connector and Smart Collector

Jump to solution

To undersand the Collectors v.s Connectors, we need to step back and look at what the SmarConnectors do.

Conceptually, the standard SmartConnectors have two main responsibilties: "Collect" raw data from various sources, and "Process" the collected data to become enriched security events and post them to a destination.

Introduced in ADP 2.30, customers can take advantage of the massive scalabilty and robustness of the Event Broker infrastructure, and move the computationaly intensive "Process" step to the highly scalable and more robust Event Broker streaming infrastructure.

This is done by using syslog Colelctors and syslog CEBs: Collectors are standalone compnents very similar to the SmartConenctors, but they only "Collect" raw syslog data like the syslog SmartConnectors do, wrap it up and post it to a dedicated eb-con-syslog topic in Event Broker. 

At that point, the Event Broker's CEB stream processors (CEB stands for Connector in Event Broker) read the data from the eb-con-syslog topic, do the parsing/normalization/enrichment/filtering processing (as the standalone SmartConnectors destination pipelines do) and post the security events on the EB topics for consumption.

In other words: as their name suggests, the syslog Collectors are lightweight component responsible for collecting raw syslog data and passing it to Event Broker for processing.

Main advantages of the new architecture:

  1. Potential for hardware consolidation and data throughput increase in the data collection layer where the Collectors are deployed: due to moving the processing to the EB streaming infrastructure.
  2. Improved stabilty and easy horizontal scalability as the data flows increase with time, or fluctuate during operations: CEBs are deployed or undeployed on the EB nodes with a single click in the ArcMC UI.
  3. Reduced network traffic due to a single data feed to Event Broker, instead of having tmultiple destinations coming from SmartConnectors
  4. The raw Syslog data is now available on the EB topic for any system that customer would like to share it with.

Note that at this time Colectors and CEBs are only available for Syslog data.

7 Replies
Micro Focus Expert
Micro Focus Expert

Re: Difference between a Smart Connector and Smart Collector

Jump to solution

I think those are part of some legacy setup, Collectors would collect data from sources (as we do with windows event connectors for example, or a fileconnector), where the connector itself is actually fetching the data instead of listening on some port.

I never seen or used these Collectors, so i do not think you need to think about needing them for any new implementations. Though i might be wrong, and if so, anyone is free to correct me :)

-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
0 Likes
alexandros_n Honored Contributor.
Honored Contributor.

Re: Difference between a Smart Connector and Smart Collector

Jump to solution

Smartcollectors are the "connectors" (or something like that) to work with Event Broker.

0 Likes
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: Difference between a Smart Connector and Smart Collector

Jump to solution

Smart connectors can also be used to connect with eventbroker.

0 Likes
evknott1 Super Contributor.
Super Contributor.

Re: Difference between a Smart Connector and Smart Collector

Jump to solution

This is from memory from a presentation at the last Protect Conference.

SmartCollector is in BETA and is part of a plan to move the parser functionality of the SmartConnectors into Event Broker.

The SmartCollector would perform the receipt of the events but do no parsing.  It would pass them to SmartConnector(s) in Event Broker where the parsing would take place.  From my understanding, this would allow us to reduce the network consumption between the SmartConnector and Event Broker if going to both Logger and ESM and allow higher EPS rates at each SmartCollector (as parsing would be handled in the Event Broker SmartConnector).

 

0 Likes
Highlighted
viktor.doundako Respected Contributor.
Respected Contributor.

Re: Difference between a Smart Connector and Smart Collector

Jump to solution

Very close. :-)

Now those compoents are generally available.

0 Likes
viktor.doundako Respected Contributor.
Respected Contributor.

Re: Difference between a Smart Connector and Smart Collector

Jump to solution

To undersand the Collectors v.s Connectors, we need to step back and look at what the SmarConnectors do.

Conceptually, the standard SmartConnectors have two main responsibilties: "Collect" raw data from various sources, and "Process" the collected data to become enriched security events and post them to a destination.

Introduced in ADP 2.30, customers can take advantage of the massive scalabilty and robustness of the Event Broker infrastructure, and move the computationaly intensive "Process" step to the highly scalable and more robust Event Broker streaming infrastructure.

This is done by using syslog Colelctors and syslog CEBs: Collectors are standalone compnents very similar to the SmartConenctors, but they only "Collect" raw syslog data like the syslog SmartConnectors do, wrap it up and post it to a dedicated eb-con-syslog topic in Event Broker. 

At that point, the Event Broker's CEB stream processors (CEB stands for Connector in Event Broker) read the data from the eb-con-syslog topic, do the parsing/normalization/enrichment/filtering processing (as the standalone SmartConnectors destination pipelines do) and post the security events on the EB topics for consumption.

In other words: as their name suggests, the syslog Collectors are lightweight component responsible for collecting raw syslog data and passing it to Event Broker for processing.

Main advantages of the new architecture:

  1. Potential for hardware consolidation and data throughput increase in the data collection layer where the Collectors are deployed: due to moving the processing to the EB streaming infrastructure.
  2. Improved stabilty and easy horizontal scalability as the data flows increase with time, or fluctuate during operations: CEBs are deployed or undeployed on the EB nodes with a single click in the ArcMC UI.
  3. Reduced network traffic due to a single data feed to Event Broker, instead of having tmultiple destinations coming from SmartConnectors
  4. The raw Syslog data is now available on the EB topic for any system that customer would like to share it with.

Note that at this time Colectors and CEBs are only available for Syslog data.

Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: Difference between a Smart Connector and Smart Collector

Jump to solution

Thanks viktor, this is very clear now !

What is the format of the data going into eb-con-syslog topic. Is it CEF or Syslog (RFC 5424).

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.