HA for Logger

HA for Logger

I split this off the connector/Logger HA feature request so we can vote separately on each.

dskeeles@hpe.co1 Absent Member.
Absent Member.

Please clarify how this would work

Honored Contributor.. DanyK7 Honored Contributor..
Honored Contributor..

Personally, I would like something along the line of:

1. have "Master" and "Slave" loggers. You configure the master, the slaves inherit the config, for example, receivers and storage groups. Such a conglomerate of loggers would be seen as a "loggers cluster"

2. On a smart, add a new destination type to "Loggers cluster". You only input the destination of the master logger, and automatically, the smart would send in loab balance/HA mode all its event to all member of the loggers cluster

3. Adding a new logger to a logger cluster would not require that I modify anything on my 100's of smarts. The new logger destination would automatically be used in the load balance/HA

4. Taking down one logger out of the cluster would cause the flow to be redistributed evenly along the remaining loggers. Not the Primary / Alternate logger schema we have now. Where an Alternate sees its events inbound flow double if the first logger is down.

5. Any search done on a logger cluster would automatically be done by default on all member of the cluster.

As Damian mention, I too would like some more info on what you meant by "Logger HA" Ofer ?

New Member.. sgremler1
New Member..

What about having Loggers in a Pool RAID type of setup, Example: 10 Loggers in a RAID 6 Cluster?  2 Servers could go down and the Logger Peering continues without errors or user intervention.  If this cannot be accomplished by HP ArcSight could we use VMware to accomplish this design and not lose performance?

zko Regular Contributor.
Regular Contributor.

Just use replicas in data storage and you're done.

Define a replication factor, and have the loggers redistribute them as needed (elasticsearch style)

Honored Contributor.. DanyK7 Honored Contributor..
Honored Contributor..

I actually like your last sentence... ElasticSearch is looking better and better everyday... 😉

pbrettle Acclaimed Contributor.
Acclaimed Contributor.

With the launch of ArcSight Investigate just around the corner, I would encourage anyone looking at HA for log storage to look closely at this. Its due for launch in late March / early April and will utilize Vertica at the back end natively. This means that it provides native clustering, redundancy, resiliency and true multi-processing support.

From a HA point of view, that means you can have a single node Vertica, but you would be reliant on RAID for resiliency of the data. But move to 3 nodes (and we arent talking massive servers here, just simple DL380's) and you can effectively use Vertica to manage the multiple replication of the data to the different nodes. It will optimize the performance too!

Really recommend looking at this going forward - and I can elaborate on this if anyone is interested.

Super Contributor.. Carl_E Super Contributor..
Super Contributor..

Will you be creating videos for Investigate sometime shortly after its release and posting them on your Youtube channel?  A walk through of the interface... how to manage Investigate.. maybe how it fits into the larger Arcsight solution (what it replaces/compliments).

I'm sure I'm not the only customer that would find this useful.

pbrettle Acclaimed Contributor.
Acclaimed Contributor.

Absolutely. I am kind of on a world tour at the moment (well, 1/2 of the world) and showing what we have to selected customers. Proof is in the eating as they say and its important to see what it does (and doesn't). Therefore yes, there will be some official ones, and some 'no so official' ones.....

I prefer to be honest and realistic - not get carried away with marketing - so expect to see something shortly. The problem is that at the moment its not released and subject to change. The sprint builds are happening thick and fast and features are getting added quick. The point of this process is that we can also roll back what doesn't work, so the target is changing constantly. Once we have a release though, its locked and we can go from there though.

Oh, and while I remember, we are working to a 3 month release cycle too! Since its all containerized, upgrades are simple and we are truly eating the DevOps model..... this yearly release cycle for a monolithic product is just too hard. Time to shift to a much faster model!

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.