Lieutenant Commander Lieutenant Commander
Lieutenant Commander
2462 views

vRealize Log Insight From VmWare parser

hi

did anybody use" vrealize log insight" events collector (it is a vmeare product for sddc environment)?

i am FWD event to my esm with syslog connector

but

all events are parsing incorrectly

i looked in the market palace as well with no results

10X

sapir
Labels (3)
0 Likes
8 Replies
Knowledge Partner Knowledge Partner
Knowledge Partner

​The versions of Log Insight i have come across so far change the header field on the forwarded syslog event so you will need to look at a parser override / flex or consider using the API instead

VMware Documentation Library

Add vRealize Log Insight Event Forwarding Destination

Ingestion API or syslog. The default value is Ingestion API (CFAPI). When events are forwarded using Ingestion API, the event's original source is preserved in the source field. When events are forwarded using syslog, the event's original source is lost and the receiver may record the message's source as the vRealize Log Insight forwarder's IP address or hostname.

Use SSL: When events are forwarded using Ingestion API, optionally secure the connection with SSL. The remote server's trust root is validated and Event Forwarding with SSL does not work with self-signed certificates installed on destination servers by default. If untrusted, import the remote server's trusted root certificate to the forwarder's keystore. See Configure vRealize Log Insight Event Forwarding with SSL.

Note

The source field may have different values depending on the protocol selected on the Event Forwarder:

a

For cfapi, the source is the initial sender's (the event originator) IP address.

b

For syslog, the source is the Event Forwarder's vRealize Log Insight instance IP address. Additionally, the syslog message text contains _li_source_path which points to the initial sender's IP address.


If you are able to post any raw events from a supported device that is not working via vRealize forwarding then we can probably see how much effort would be required to fix?

0 Likes

Facing the same issue here and here's what I have reached so far:

This is the raw message that arrives from ESXi which is supported

<1>2017-05-21T08:40:20.664Z AB-CDE1-ESX1 vmkernel: cpu1:36106)World: 14302: VC opID hostd-0ff5 maps to vmkernel opID 4822e778

This is the raw message that arrives from vRealize which is not supported

<1>May 21 11:38:45 10.1.1.1 1 2017-05-21T08:40:20.664Z AB-CDE1-ESX1 vmkernel: cpu1:36106)World: 14302: VC opID hostd-0ff5 maps to vmkernel opID 4822e778

I don't want to write a complete Flex override, my idea is to find a way to strip the message header from vRealize and send it to a syslog connector which will be able to parse it correctly as vmware

I was able to build a flex to separate the vRealize message into tokens, and extract the exact message part that is parsable by the normal syslog connector, but I didn't figure out how to send this specific token to another syslog :D, Do you think that the smart connector have this functionality?

Another idea, thinking outside the smart connector funtionality, is to find a software that receives syslog from vRealize and have the capability to change the headers, I was reading about "rsyslog /syslogng", but didn't complete my research yet.

Mustapha
0 Likes

Hi,

I got that fixed. After introducing rsyslog as a front layer. ESXi events are being parsed as expected. All I need to be done is ask rsyslog to pass on the "message" part of the syslog event to the smart connector.

Mustapha
0 Likes
Lieutenant Commander
Lieutenant Commander

Hi can you help over it,

 

what type of fiter you mention so it parsed well

0 Likes

Hi,

The goal is to extract the "message" part and pass it through to the smart connector.

Check my previous reply, I pasted two samples that shows how the issue need to be fixed.

in rsyslog, you will need to use templates to reformat the event and sent it, should look something like this.

https://www.rsyslog.com/doc/v8-stable/configuration/templates.html

template(name="ForwardFormat" type="list") {
    constant(value="")
    property(name="msg" spifno1stsp="on" )
    property(name="msg")
    }

HTH

Mustapha
0 Likes
Lieutenant Commander
Lieutenant Commander

can you help with syslog-ng

0 Likes

Example template took it from syslog-ng documentation:

https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.20/administration-guide/59

template t_demo_filetemplate {
    template("${MESSAGE}\n");
};
destination d_file {
    file("/var/log/messages" template(t_demo_filetemplate));
};

Something similar should fix the issue.

This is what would I have done to test my template:

1. Take the "message" part of the event and push it directly to the syslog connector (echo "my message syslog formated event" | nc 127.0.0.1 514

2. once you find which part will be parsed correctly by the connector, the next step is to figure the correct template in syslog-ng

3. play around with the syslog-ng template and ask it to write to file to understand how the it's behaving untill you find the correct template that will push the message as required.

HTH

Mustapha
0 Likes

Hello, recently we have connected loginsight to arcsight, I had to write flex syslog subparser, because loginsiht is inserting custom tag to message...On the end, We have fid out, taht loginsight is also replacing one string of message with tag, so there are eg. username missing....VMWare resolution is to use Ingestion API instead...would you know, if there is a way to (quoting vmware) "configure Ingestion agent in Arcsight"?
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.