Highlighted
Absent Member.
Absent Member.
309 views

Ciscoworks feed parsing issue


Sentinel 6.1 will not parse my feed from Ciscoworks. All network syslog
data is collected by ciscoworks and stored in a file syslog.log. This
file can be pulled by the collector manager as a file source or pushed
via Snare to the collector mgr.. the data cannot be parsed using either
method using standard Cisco Collector (Cisco IOS Router xx Cisco Switch
and Router 6.1r2).
Is there a working collector or do I have to build one?


--
eisensee
------------------------------------------------------------------------
eisensee's Profile: http://forums.novell.com/member.php?userid=98444
View this thread: http://forums.novell.com/showthread.php?t=425629

0 Likes
3 Replies
Highlighted
Absent Member.
Absent Member.

Re: Ciscoworks feed parsing issue


Hello eisensee,

So there are a number of places where the standard Cisco stuff can
break down:

1) The source device has to be configured exactly as documented in the
relevant Collector documentation. Cisco allows you to do a lot of crazy
things to your syslog messages, some of which violate the Syslog RFC
standard. So you need to make sure that the data produced by the
original source device is 'clean'.

2) You then send the data to CiscoWorks - in our past analyses of this
product, it pretty much just passively collects the data and can forward
as needed. Again, there may be configuration options that might cause it
to mess up the standard syslog formatting that you need to pay attention
to. I should also note that a few years ago we had a customer using
CiscoWorks and they discovered SEVERE performance problems with it.

3) You then say that Cisco dumps the data to file - is there a reason
you don't just configure it to forward the syslog events directly to
Sentinel? When syslog writes to file, there's no standard as to how the
events are stored - syslog is ONLY a wire standard. As such, you
probably want to avoid that if possible. IIRC, configuring CiscoWorks to
forward syslog data is pretty easy.

4) Once the data is in a file, there is (as I said) no standard as to
how it is stored. Correspondingly, the method you use to read it from
the file (e.g. Snare) may or may not mess up the data further. All bets
are off, to some extent.

Now, if you can't simply forward CiscoWorks to Sentinel, and have to
deal with the file format, but you can figure out how the format is
being munged (we follow the strict syslog RFC standard), then you can
probably add a simple pre-parser to the existing Collector to "undo" the
munged data. If you post a dump (Connector Dump) of the data as you
receive it, I can probably provide some simply guidance.


--
DCorlette
------------------------------------------------------------------------
DCorlette's Profile: http://forums.novell.com/member.php?userid=4437
View this thread: http://forums.novell.com/showthread.php?t=425629

0 Likes
Highlighted
Absent Member.
Absent Member.

Re: Ciscoworks feed parsing issue


Hello dcorlette,
Wow we're getting the heavy hitters! I like it!
1) This issue surfaced after the upgrade from 6.0 to 6.1. This used to
to parse correctly, so I'm running under the assumption that the data is
formatted in an acceptable fashion.

2) CiscoWorks is designed (as I've been told) to be an end point
collection system and not a relay to further systems. It keeps the data
in a rotating file (FIFO) as the file syslog.log (simple text file). In
our system we >1000 devices report to ciscoworks (switches, APs,
routers).

3) Prior to the upgrade, I simply performed a file read and it all
parsed correctly. I tried that same arrangement and the collector no
longer parsed the data correctly. With the thought that a push might
work better than a pull, I employed the application Epilog (What Snare
is for Event logs, Epilog does for text files) to push the data as a
syslog stream. This yielded the same results.

Here is the first entry in a connector dump:
{"s_AppId":"sm-ciscowks.smad2.savemart.com","i_syslog_priority":"13","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"182","s_Process":null,"CONNECTION_MODE":"map","s_RV25":"F6FEB664-D018-102D-AD76-001A6450619A","s_RV24":"DD9ACEA3-C5A7-102D-BDF1-001A6450619A","i_Type":"0","i_Second":"54","s_Version":"6r9","s_RXBufferString":"Nov
11 15:30:54 10.101.251.4 sm-ciscowks.smad2.savemart.com\t\t0\tNov 11
15:30:17 172.253.248.33 2182635: 418971: Nov 11 15:30:14.739:
\/\/426174\/801DF193ED75\/CCAPI\/cc_api_call_connected:","s_Body":"sm-ciscowks.smad2.savemart.com\t\t0\tNov
11 15:30:17 172.253.248.33 2182635: 418971: Nov 11 15:30:14.739:
\/\/426174\/801DF193ED75\/CCAPI\/cc_api_call_connected:","i_milliseconds":"1289518254194","s_raw_message2":"sm-ciscowks.smad2.savemart.com\t\t0\tNov
11 15:30:17 172.253.248.33 2182635: 418971: Nov 11 15:30:14.739:
\/\/426174\/801DF193ED75\/CCAPI\/cc_api_call_connected:","s_MessageOriginatorPort":"36305","i_Minute":"30","s_Date":"Nov
11
15:30:54","i_TrustDeviceTime":"","i_DayOfMonth":"11","i_Year":"2010","s_SyslogRelayIp":"10.101.251.4","s_MessageOriginatorHost":"10.101.251.4","s_Pid":null,"i_Month":"10","i_syslog_facility":"1","i_syslog_severity":"5"}


--
eisensee
------------------------------------------------------------------------
eisensee's Profile: http://forums.novell.com/member.php?userid=98444
View this thread: http://forums.novell.com/showthread.php?t=425629

0 Likes
Highlighted
Absent Member.
Absent Member.

Re: Ciscoworks feed parsing issue


OK, hmm...

1) At issue is more likely the Collector version, not so much the
platform version. The old Collectors made an attempt at correcting some
weird syslog formats, but the feature ended up being too confusing and
error prone for us to support. The new Collectors only support proper
RFC-compliant syslog, for a number of reasons which we can get into if
you like.

2) I may be wrong, but I believe a customer told me that latter-day
versions of CiscoWorks introduced a syslog forwarding option. I poked
around on Cisco for a bit, but there are hundreds of specific products
with the 'CiscoWorks' label so I don't know what applies.

3) OK, so here's the deal. RFC-compliant syslog message are constructed
as:
MMM DD HH:MM:SS hostid message
The original message from your switch should look something like:
Nov 11 15:30:17 172.253.248.33
\/\/426174\/801DF193ED75\/CCAPI\/cc_api_call_connected:

It actually looks like (guessing a bit, here:
Nov 11 15:30:17 172.253.248.33 2182635: 418971: Nov 11 15:30:14.739:
\/\/426174\/801DF193ED75\/CCAPI\/cc_api_call_connected:

Which indicates that a couple numbers and another timestamp are being
injected into the "message" portion. May or may not be an issue, if the
Collector handles it that way - review the Collector doc for proper
configuration details.

But then your Epilog gets a hold of the message, and inserts *another*
header:
Nov 11 15:30:54 10.101.251.4 sm-ciscowks.smad2.savemart.com\t\t0\tNov
11 15:30:17 172.253.248.33 2182635: 418971: Nov 11 15:30:14.739:
\/\/426174\/801DF193ED75\/CCAPI\/cc_api_call_connected:

This violates RFC3164 in a number of ways, namely:
- it's not supposed to modify the original message IN ANY WAY if it's
already a proper syslog message (of course, Epilog may assume it's NOT a
syslog message, since it's in a file)
- It's using a fully-qualified hostname in its header, which is NOT
VALID
- There are tab characters after the header, which aren't proper syslog
characters

If I couldn't configure Epilog to not be stupid, what I would do is
create a little 'custom.js' script, and define my customerPreparse()
method to strip off the entire ugly Epilog header, something like:
Record.prototype.customPreparse = function() {
this.s_RXBufferString =
this.s_RXBufferString.substr(this.s_RXBufferString.lastIndexOf("\t"));
}

(You may need to do the same thing to rec.s_Body as well, and note that
I haven't tested this code at all!).

Then just follow the normal process to inject custom.js into your
Collectors, set the Execution Mode to 'custom', and you'll be up and
running.

NOTE: the major thing that Epilog is breaking here, however, is not the
Collector - the Syslog Connector also does some minimal parsing of the
input and will automatically create Event Source nodes based on the
syslog header. The syslog header is supposed to list the hostid of the
ORIGINAL event source as it's second element (after the timestamp), and
by injecting the ciscoworks device hostid, Epilog breaks that (in our
parlance, that's the Reporter, not the Observer).

If you look directly at the file on the CiscoWorks device, what does it
look like? Everything after the \t\t0\t in the sample you sent?

You might ditch Epilog entirely and use something like tail -F or
'netcat' instead, which won't add silly stuff to the messages.


--
DCorlette
------------------------------------------------------------------------
DCorlette's Profile: http://forums.novell.com/member.php?userid=4437
View this thread: http://forums.novell.com/showthread.php?t=425629

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.