Anonymous_User Absent Member.
Absent Member.
299 views

Data Maps & Custom Collectors


Hi Guys,

I am fairly new to the development side of the collectors for Sentinel,
and have what may be a total beginners question but I've tried the docs
and feel I am getting no where with an answer so hopefully you can help
out 🙂

So I am constructing an event from within the collector, I have read
that its best practice to try to use maps where possible so I am trying
to do this. If I hard code for example e.TargetUserID = <the parsed
string that I have>, then that works, but I want to try to make use of
the Rec2Evt.map as most of the data that I am populating at this point
is listed in here.

What I have done is to add into the the following:


Code:
--------------------

Collector.prototype.initialize = function() {
this.MAPS.Rec2Evt = new DataMap(this.CONFIG.collDir + "/Rec2Evt.map");

...
}

--------------------


Then within parse have the following:


Code:
--------------------

...
rec.testIP = "123.4.5.6";
...
rec.convert(this, instance.MAPS.Rec2Evt);
instance.SEND_EVENT = true;
return true;
}

--------------------


Within the Rec2Evt.map file it has the default list of Sentinel Event
Fields and I have appended a record for TargetIP: TargetIP,testIP

Have I missed any obvious steps out? What I was expecting to happen was
when the event is recieved and parsed in Sentinel the TargetIP field
should have the value 123.4.5.6, when I look in either the ESM or the
Sentinel 7 webUI I dont see this field getting set, other fields which I
manually set are being set correctly.

This is the first time that I have tried to use the data maps so I
assume I am doing something wrong and any pointers you guys have would
be great,

Thanks


--
alanforrest
------------------------------------------------------------------------
alanforrest's Profile: http://forums.novell.com/member.php?userid=90508
View this thread: http://forums.novell.com/showthread.php?t=453791

0 Likes
5 Replies
Anonymous_User Absent Member.
Absent Member.

Re: Data Maps & Custom Collectors


alanforrest;2184438 Wrote:
> Hi Guys,
>
> I am fairly new to the development side of the collectors for Sentinel,
> and have what may be a total beginners question but I've tried the docs
> and feel I am getting no where with an answer so hopefully you can help
> out 🙂
>
> So I am constructing an event from within the collector, I have read
> that its best practice to try to use maps where possible so I am trying
> to do this. If I hard code for example e.TargetUserID = <the parsed
> string that I have>, then that works, but I want to try to make use of
> the Rec2Evt.map as most of the data that I am populating at this point
> is listed in here.
>
> What I have done is to add into the the following:
>
> >

Code:
--------------------
> >

> Collector.prototype.initialize = function() {
> this.MAPS.Rec2Evt = new DataMap(this.CONFIG.collDir + "/Rec2Evt.map");
>
> ...
> }
>

--------------------
> >

>
> Then within parse have the following:
>
> >

Code:
--------------------
> >

> ...
> rec.testIP = "123.4.5.6";
> ...
> rec.convert(this, instance.MAPS.Rec2Evt);
> instance.SEND_EVENT = true;
> return true;
> }
>

--------------------
> >

>
> Within the Rec2Evt.map file it has the default list of Sentinel Event
> Fields and I have appended a record for TargetIP: TargetIP,testIP
>
> Have I missed any obvious steps out? What I was expecting to happen
> was when the event is recieved and parsed in Sentinel the TargetIP
> field should have the value 123.4.5.6, when I look in either the ESM
> or the Sentinel 7 webUI I dont see this field getting set, other
> fields which I manually set are being set correctly.
>
> This is the first time that I have tried to use the data maps so I
> assume I am doing something wrong and any pointers you guys have would
> be great,
>
> Thanks


Hey Alan,

If all you are trying to do is set an input record field to a sentinel
field I think it is easier then this.

Add the line to Rec2Evt.map like you have, and then place your value in
this.testIP in your parser. You do not need to call convert or define
the map in this case.



Now if you were making a new custom map you would need to define the
map.

In a new map where you had a mapfile something like

evtcode,eventname

You would need to setup the map, and then call it. Use the syslog
severity maps as an example.

this.MAPS.mapname = new KeyMap(instance.CONFIG.collDir +
"mapfile.map");

Then when you wanted to place the value in a field you would just use
this.eventname = instance.MAPS.mapname.lookup(evtcode)


--
cslye
------------------------------------------------------------------------
cslye's Profile: http://forums.novell.com/member.php?userid=26909
View this thread: http://forums.novell.com/showthread.php?t=453791

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Data Maps & Custom Collectors


Hi Mate,

Thanks for the reply, it is now working as expected 🙂

> You do not need to call convert or define the map in this case.


Just took the define and convert calls out of my code and its working.
It looks like the following line was causing it to either not load the
correct map or to ignore the default map as when I copy this line back
in it stops working:


Code:
--------------------

this.MAPS.Rec2Evt = new DataMap(this.CONFIG.collDir + "Rec2Evt.map");

--------------------


Thanks Again,

Alan


--
alanforrest
------------------------------------------------------------------------
alanforrest's Profile: http://forums.novell.com/member.php?userid=90508
View this thread: http://forums.novell.com/showthread.php?t=453791

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Data Maps & Custom Collectors


Yes, just to clarify, loading of the default "Rec2Evt.map" is handled by
the template, so you don't have to deal with it. In more complex
scenarios you could create several different maps and selectively apply
them, but 99% of the time all you have to do is parse out the data into
a 'rec' attribute, then list that attribute in the Rec2Evt.map file.

One note, Alan, when you're in the parse() method the scope is actually
that of the 'rec' object itself, so you can refer to 'rec' as 'this'
(and in fact it's safer to do so). For example:

Record.prototype.parse = function() {
...
this.testIP = "123.4.5.6";
...
}

Hope this helps, and let us know if you have any other questions!


--
DCorlette
------------------------------------------------------------------------
DCorlette's Profile: http://forums.novell.com/member.php?userid=4437
View this thread: http://forums.novell.com/showthread.php?t=453791

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Data Maps & Custom Collectors


Excellent, yeah the mapping stuff is all working great thanks.

So in terms of if I had 3 or 4 attributes, is it ok to add these to the
Rec2Evt.map rather than create a second map? I wasn't sure if this was
recommended or whether to add these to a new map. At the moment I have
these added to a new map, I thought about adding them to the Rec2Evt but
I got the impression that this list of attributes shouldn't really be
modified other than the second column.

Cool, Yeah I have tried to refer to the record where possible as "this"
in my code since tidying it up. 🙂

Thanks again for your replies.


--
alanforrest
------------------------------------------------------------------------
alanforrest's Profile: http://forums.novell.com/member.php?userid=90508
View this thread: http://forums.novell.com/showthread.php?t=453791

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Data Maps & Custom Collectors


Hi Alan,

I'm not quite sure what you mean by "3 or 4 attributes", but here are
some guidelines:

Part of the Collector development process is to make a best-effort
attempt to parse out semantically distinct fields from the input and map
them to the Sentinel schema in a normalized way. Sometimes this is easy
- there's an IP address that's the target of a connection, extract it
and map it to TargetIP (TargetIP should already exist in Rec2Evt.map and
you just need to list the 'rec' attribute into which you parsed that
target IP). Sometimes this requires a little more work, for example
timestamps and whatnot that need normalization. Sometimes this is really
tricky, and you can't find a nice match to a Sentinel schema field.
Let's break this down into the following categories:

1) Simple 1:1 matches, like the IP address example above

2) 1:N matches, where you need to subparse a bit. An example might be a
path like C:\WINDOWS\system32\etc\hosts; this would map to
TargetDataName = 'hosts', TargetDataContainer = '/windows/system32',
TargetDataNamespace = 'c' (note that since Windows is case-insensitive,
everything has been lowercased and the path separators normalized - we
provide some utility flags and methods for this in the latest SDK which
will be out soon.

3) Mapped matches: in this scenario, you have a field maybe that
indicates severity using some arbitrary proprietary scale, and you need
to map this to Sentinel's 0-5 Severity. In this case it's good to use a
KeyMap, put all your possible input values in the LHC, and then map them
to Sentinel Severities in the RHC. Then you can use lookup() to look up
your input and map it to the correct output, put that output in a 'rec'
attribute, and then list that attribute in Rec2Evt.map (in this example
on the RHS after 'Severity,'

4) No schema match, doesn't need to be correlated: An example here
might be "session type", which is something that Windows provides but
that we don't (yet) have a dedicated schema field for (although we are
considering it). Let's say you want to record that information in the
event, but you don't need to correlate on that value. In that case you
can use the 'add2EI()' method to add an JSON NVP to the
ExtendedInformation field, something like 'LoginType: interactive'.

5) No schema match, need to correlate: This is the trickiest case,
where you can't find a place to put your data but you need it in a
separate field so you can correlate on it. For this scenario you can use
one of the many unallocated ReservedVarXX fields. What you need to do is
pick an unused field, add it to Rec2Evt.map, and map your data to it.
The trick is that you can't guarantee that some other Collector is not
using that field for a different purpose, so you have to be a bit more
careful when writing correlation rules etc to filter for your data
only.

In other words, the only attributes you should ever be adding to
Rec2Evt.map are ReservedVar fields. BTW, the event schema is fully
documented here:'Sentinel Event Schema'
(http://www.novell.com/developer/plugin-sdk/event_schema.html)
but note that not all fields are present in all platforms.


--
DCorlette
------------------------------------------------------------------------
DCorlette's Profile: http://forums.novell.com/member.php?userid=4437
View this thread: http://forums.novell.com/showthread.php?t=453791

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.