New Ranks & Badges For The Community!
Notice something different? The ranks and associated badges have gone "Star Fleet". See what they all mean HERE
Highlighted
Lieutenant
Lieutenant
1602 views

Integrating OS logs from cloud environment VMs

Hello,

I am looking for some guidance/recommendations for onboarding OS logs(Windows & *NIX) from AWS and Azure environments into on-prem ArcSight:

I have a heterogeneous cloud environment(AWS& Azure) with many subscriptions/accounts that are not joined to the domain. These environment contain both Windows and *nix hosts VMs and we are trying to find a good approach to integrate the logs from these VMs with our on-prem ArcSight.

These are our challenges:

  • More than 50 Azure subscriptions and AWS accts
  • Accts/subscriptions belong to the different units
  • The environments are not domain joined
  • They are also not connected to each other(all standalone environments)
  • A VPN from the cloud provider to the on-premises site is not an option at this point.

Current approach:

  • Current approach is to configure each linux server to forward logs to a Unix connector in each account/subscription.Connector then sends logs to on-prem Arcsight
    • However, that means a syslog connector(aggregation point) will be required for each account/subscription)
    • Not ideal from a management perspective
  • Since the environments are not domain joined, using a WUC with a service account performing WMI calls to pull logs will not be an option
  • Wondering whether I can leverage Windows event forwarding
  • We may be able to leverage scripting for potential configuration/auditing changes needed

Question:

  • Do I need to set up a connector for each subscription or is there a way to group them so I don't have over  50+ connectors(for both Windows and Linux) to manually set up and maintain? 

These are the main questions I have as of now. I am looking to see if anyone might have had any experience in a situation like this before (basically getting connectors setup to non-domain hosts being sent over the Internet).

Thanks for the help.

Labels (2)
0 Likes
7 Replies
Highlighted
Fleet Admiral
Fleet Admiral

Ok, lots of questions and points there, but there are a number of things to address.

Some initial comments though:

1) I am a big fan of Windows Event Forwarding (WEF). It is a simple, scalable and straightforward way to obtain log data. There are a number of advantages here in that you can use the NATIVE communications between Windows machines and bring the data to a central point - then collect the data from that server (either as WEF data natively or via an SQL conenction). Its easy, simple to enforce and since its native Windows communications, its easy to setup and manage. I would encourage anyone looking at Windows platforms to take a closer look at WEF.

2) You mention about getting the events from the platform also - this is an option that isnt widely published I am afraid, mainly because its pretty new. Azure as a platform does have the ability to deliver the data via this method and I would encourage anyone to take a closer look here:

Integrate logs from Azure resources into your SIEM systems | Microsoft Docs

You will see that we can collect the data via JSON -  though you will need to write a FlexConnector to process this. Additionally, there is a new capability that Microsoft provides called AzureSIEM or AzSIEM. Its new and partly hidden, but expect it to be a lot more integrated in the coming months. The information about it has leaked out and you can find it here:

https://azsiempublicdrops.blob.core.windows.net/drops/Azure%20SIEM%20Integrator%20User%20Guide.pdf

Take a look at the guide and it will cover off the way to integrate ArcSight SmartConnectors to pull and process the data. This means that you can receive the data through the Azure platform, rather than via the native Windows mechanisms. You will need the Azure SIEM parser for this also:

https://marketplace.saas.hpe.com/arcsight/content/azure-azsiem-parser

3) For Amazon, its a little easier since there is a mechanism that is available to do this, but at the moment, its pretty focused around S3 platforms rather than others. Expect to see this develop pretty quickly going forward, but take a look here:

Some generic information here:

https://www.protect724.hpe.com/message/61032?commentID=61032#comment-61032

And Richard Kent very kindly put together some information here:

Not sure I have answered your question there, but hopefully this is a start!

Highlighted
Captain Captain
Captain

Hello,

Is this still the sollution? using the AZSIEM (or AZLOG as it is called these days).
Using a JSONFlexConnector, we need parser files, Does anyone want to share the files they have?

I've been looking in using the WiNC Connector Azure parser, showin in the Marketplace, but i don't know if this is an old solution or a newer one.

Anyone care to share their thoughts?

 

Roy

0 Likes
Highlighted
Commodore Commodore
Commodore

What's the latest on this - we're running into a similar scenario

0 Likes
Highlighted
Lieutenant Commander
Lieutenant Commander

Hey Paul, Did microsoft rename Azsiem to Azlogs?

0 Likes
Highlighted
Fleet Admiral
Fleet Admiral

Seems they did - things are moving fast with Microsoft around this - so yeah, looks like it is:

Download Microsoft Azure Log Integration from Official Microsoft Download Center

0 Likes
Highlighted
Captain
Captain

Invaluable information here, Thanks Paul.

0 Likes
Highlighted
Lieutenant
Lieutenant

Thanks a tonne Paul. My apologies for taking awhile to respond. Thanks for the detailed reply.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.