ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins. Read more for important details.
ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins.Read more for important details.

CEF/ ESM Schema extention to support cloud solutions

Idea ID 2810474

CEF/ ESM Schema extention to support cloud solutions

CEF standard was developed ages ago. Nowadays, there are so many cloud-based log sources which require additional fields. Most of the time we use custom/flex strings, but the number of those are limited and it becomes a mess to maintain all of these across the board. It is time to consider to issue a new CEF version.

When we consume AWS, GCP, Azure logs we have such entities as:

- account id | tenant id (I mean AWS account id, Google Cloud org Id, or other unique identifier.)

- availability zone

- storage bucket (S3, storage, blob)

- container name/id

 - instance name (which may be not the same  as deviceHostName or destinationHostName)

 - instance id

- region

 - user type (root/ iam )

 

None of these have been properly represented in the schema since the cloud era began to spread. Therefore, random fields like fileId (access key ID) or filePath (full user path) are used to store some of the IDs or role names and so on.

2 Comments
Micro Focus Contributor
Micro Focus Contributor
Status changed to: Accepted

The ArcSight ecosystem is moving towards having AVRO as it base schema format.  AVRO has many features that neither ESM_Binary nor CEF currently support, such as schema flexibility and schema registry to ensure compatibility with different schema versions.

The July 2020 release will be the first release to embrace AVRO and future releases will continue to enhance this technology.

Vice Admiral
Vice Admiral

Dear @dalesio , 

Is such movement going to enhance the current CEF and use AVRO as the schema format?

I do think that makes sense, but what about also providing the CEF Schema Registry that you can use in conjunction with any Kafka Cluster AND with the ArcSight Products?

https://community.microfocus.com/t5/ArcSight-Idea-Exchange/CEF-Schema-Registry/idi-p/

Regards,

Karl.

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.