DB Collector does not show events


Hello folks,

Any advice on the following? I'm writing a DB collector, but it doesn't
output events. The raw data is stored as it should.

The sqlquery.base is ok. Besides that the only modifications are shown
below:

in
Collector.prototype.initialize = function(){
....
this.PARSER.getOffsetData = function(input){
if (typeof input.RXMap != "undefined") {
return input.RXMap.col_entryNr;
}
}
conn.addParser(this.PARSER.getOffsetData);
}

and in
Record.prototype.parse = function(e){
....
First setting variable DateTime, which is okay and used in the
following:
....
if (e.setObserverEventTime(DateTime)) {
this.evtName = "Example event";
this.msg = this.RXMap.col_text1 " " this.RXMap.col_text2 "
by " this.RXMap.col_userID;
// set SEND_EVENT to true if your parsing logic worked correctly
instance.SEND_EVENT = true;
}
// If you can't parse...
else {
rec.sendUnsupported();
}
}

What am I missing here?
I get the following error in the server0.0.log and I've got no idea
where to dig.. To me everything looks ok in the debug mode.


Code:
--------------------

Tue Jun 24 17:22:46 EEST 2014|SEVERE|Collector [EXAMPLE APPLICATION2]|esecurity.base.util.logging.UnexpectedExceptionHandler.handle
An unexpected exception occurred while saving offset for device undefined.; Exception UUID has to be represented by the standard 36-char or 32-char hex representation - uuid = undefined; java.lang.NumberFormatException;
Tue Jun 24 17:22:46 EEST 2014|SEVERE|Collector [EXAMPLE APPLICATION2]|esecurity.base.util.logging.UnexpectedExceptionHandler.handle
java.lang.NumberFormatException: UUID has to be represented by the standard 36-char or 32-char hex representation - uuid = undefined
at com.esecurity.uuid.UUID.<init>(UUID.java:128)
at esecurity.base.datamodel.attribute.EsecUuid.setValue(EsecUuid.java:64)
at esecurity.base.datamodel.attribute.EsecUuid.<init>(EsecUuid.java:36)
at esecurity.db.object.EventSourceOffsetImpl.setEventSourceId(EventSourceOffsetImpl.java:35)
at esecurity.ccs.comp.proxycollector.common.ProxyDeviceOffsetManager.saveOffset(ProxyDeviceOffsetManager.java:195)
at esecurity.ccs.comp.proxycollector.common.ProxyManager.setDeviceOffset(ProxyManager.java:1020)
at esecurity.ccs.comp.proxycollector.common.CollectorManager$ResultsHandler.setDeviceOffset(CollectorManager.java:1505)
at esecurity.ccs.comp.proxycollector.common.PluggableEngineMediator.setOffset(PluggableEngineMediator.java:144)
at esecurity.ccs.comp.evtsrcmgt.collector.util.ScriptEngineContext.setOffset(ScriptEngineContext.java:210)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:155)
at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:243)
at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:76)
at org.mozilla.javascript.gen.c1._c318(Unknown Source)
at org.mozilla.javascript.gen.c1.call(Unknown Source)
at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:66)
at org.mozilla.javascript.gen.c1._c34(Unknown Source)
at org.mozilla.javascript.gen.c1.call(Unknown Source)
at org.mozilla.javascript.optimizer.OptRuntime.callProp0(OptRuntime.java:119)
at org.mozilla.javascript.gen.c1._c0(Unknown Source)
at org.mozilla.javascript.gen.c1.call(Unknown Source)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:393)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2834)
at org.mozilla.javascript.gen.c1.call(Unknown Source)
at org.mozilla.javascript.gen.c1.exec(Unknown Source)
at esecurity.ccs.comp.evtsrcmgt.collector.util.BSFEngine.runRhino(BSFEngine.java:121)
at esecurity.ccs.comp.evtsrcmgt.collector.util.BSFEngine.run(BSFEngine.java:65)
at esecurity.ccs.comp.proxycollector.common.SingleThreadCollectorExecutor$1.runEngine(SingleThreadCollectorExecutor.java:232)
at esecurity.ccs.comp.proxycollector.common.SingleThreadCollectorExecutor$1.run(SingleThreadCollectorExecutor.java:149)
at java.lang.Thread.run(Unknown Source)

--------------------


I've done DB collectors couple of years back and had problems then too,
only different ones..
Help will be appreciated.

BR,
Petri


--
petri_vesamaki
------------------------------------------------------------------------
petri_vesamaki's Profile: https://forums.netiq.com/member.php?userid=379
View this thread: https://forums.netiq.com/showthread.php?t=51166


  • Forgot to mention Rec2Evt.map which has the following:

    EventName,evtName
    Message,msg

    There seems to be other errors as well in the server0.0.log, they seem
    to vary between the attempts.

    Code:
    --------------------

    Wed Jun 25 10:20:43 EEST 2014|SEVERE|collector-read-thread-undefined|esecurity.base.ccs.proxy.ContainerProxy$1.uncaughtException
    Uncaught exception; Exception Wrapped java.lang.InterruptedException; org.mozilla.javascript.WrappedException; ; Caused by java.lang.InterruptedException;
    Wed Jun 25 10:20:43 EEST 2014|SEVERE|collector-read-thread-undefined|esecurity.base.ccs.proxy.ContainerProxy$1.uncaughtException
    org.mozilla.javascript.WrappedException: Wrapped java.lang.InterruptedException
    at org.mozilla.javascript.Context.throwAsScriptRuntimeEx(Context.java:1757)
    at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:170)
    at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:243)
    at org.mozilla.javascript.optimizer.OptRuntime.callProp0(OptRuntime.java:119)
    at org.mozilla.javascript.gen.c1._c32(Unknown Source)
    at org.mozilla.javascript.gen.c1.call(Unknown Source)
    at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:393)
    at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2834)
    at org.mozilla.javascript.gen.c1.call(Unknown Source)
    at org.mozilla.javascript.InterfaceAdapter.invokeImpl(InterfaceAdapter.java:147)
    at org.mozilla.javascript.InterfaceAdapter$1.run(InterfaceAdapter.java:122)
    at org.mozilla.javascript.Context.call(Context.java:577)
    at org.mozilla.javascript.ContextFactory.call(ContextFactory.java:503)
    at org.mozilla.javascript.InterfaceAdapter.invoke(InterfaceAdapter.java:125)
    at org.mozilla.javascript.jdk13.VMBridge_jdk13$1.invoke(VMBridge_jdk13.java:136)
    at com.sun.proxy.$Proxy8.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
    Caused by: java.lang.InterruptedException
    at java.util.concurrent.SynchronousQueue.take(Unknown Source)
    at esecurity.ccs.comp.proxycollector.common.CollectorManager$PluggableEngineConnector.readData(CollectorManager.java:1300)
    at esecurity.ccs.comp.proxycollector.common.PluggableEngineMediator.readData(PluggableEngineMediator.java:79)
    at esecurity.ccs.comp.evtsrcmgt.collector.util.ScriptEngineContext.readData(ScriptEngineContext.java:198)
    at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:155)
    ... 15 more

    Wed Jun 25 10:20:44 EEST 2014|WARNING|TimerThreadPool pool|esecurity.ccs.comp.evtsrcmgt.collector.util.ScriptEngineUtil.setStopFlag
    There is no context match the uuid: 853DA326-D455-1031-B752-000C292A3FE8

    --------------------


    The uuid in the end corresponds to to collectors uuid.


    --
    petri_vesamaki
    ------------------------------------------------------------------------
    petri_vesamaki's Profile: https://forums.netiq.com/member.php?userid=379
    View this thread: https://forums.netiq.com/showthread.php?t=51166

  • Wild guesses:

    Have you tried using this in a mode other than collecting from a database,
    for example by getting events from a file connector?

    I presume you started with the SDK and built up from nothing rather than
    pulling any code in from an existing collector made to work with
    databases. Have you tried doing the opposite in case there is something
    working collectors do that you are missing?

    Does using the debugger with a working collector made for DBs look
    substantially similar at the end of the event's parsing to yours? Are the
    events you are sending in becoming unsupported events (I presume this is
    the case since I do not see any logic telling the system otherwise) and
    have you tried setting the flags so that the collector sends it as
    supported? I would expect sendUnsupported() to work but haven't done a DB
    collector in a while.

    --
    Good luck.

    If you find this post helpful and are logged into the web interface,
    show your appreciation and click on the star below...

  • Okay, some more findings..
    At least the first error seems to occur when the offset of the Event
    Source is modified, or the Event Source is restarted. So it doesn't seem
    to have any relation to the problem with sending event. And anyways the
    offset handling still seems to work.

    When tracking the record handling in the ESM Collector debug-mode,
    everything seems to be ok until reaching the
    Collector.CONFIG.scriptContext.fireEvent function.
    This function seems to do nothing at all and its contents are as
    following:

    Code:
    --------------------

    function fireEvent() {/* void fireEvent(esecurity.db.object.Event) void fireEvent(esecurity.ccs.comp.evtsrcmgt.collector.util.EventData,esecurity.ccs.comp.evtsrcmgt.connectorapi.ConnectorData) */}

    --------------------

    So everything is commented out. Is this the way it should be?


    --
    petri_vesamaki
    ------------------------------------------------------------------------
    petri_vesamaki's Profile: https://forums.netiq.com/member.php?userid=379
    View this thread: https://forums.netiq.com/showthread.php?t=51166


  • Thanks for your reply ab,
    You're right that started writing from scratch, however I have done a
    lot of comparison to existing DB collectors.
    I didn't try with other connector nor other db collectors, maybe I'll
    try tomorrow.
    I tried using the sendUnsupported(), but the result is the same.
    Also added a severity mapping, but that had no effect.
    Are there other flags to set besides of SEND_EVENT ?


    --
    petri_vesamaki
    ------------------------------------------------------------------------
    petri_vesamaki's Profile: https://forums.netiq.com/member.php?userid=379
    View this thread: https://forums.netiq.com/showthread.php?t=51166


  • Finally got it working with the help of NetIQ support. Thank you.

    So the problem was with setting the date. The debugger showed no errors
    and displayed the time correctly, but under the hood there was something
    wrong, which I don't quite understand.

    This is how it didn't work:

    Code:
    --------------------

    Record.prototype.parse = function(e){
    ...
    // The DateYYYY etc. are set from two columns in the table, nothing special about them
    var eventTime = new Date(DateYYYY,DateMM,DateDD,TimeHH,TimeMM,TimeSS);

    ...
    if (e.setObserverEventTime(DateTime)) {
    this.evtName = "Example event";
    this.msg = this.RXMap.col_text1 " " this.RXMap.col_text2 " by " this.RXMap.col_userID;
    // set SEND_EVENT to true if your parsing logic worked correctly
    instance.SEND_EVENT = true;
    }
    // If you can't parse...
    else {
    rec.sendUnsupported();
    }
    }

    --------------------


    And this is how it works:


    Code:
    --------------------

    Record.prototype.parse = function(e){

    if (e.setObserverEventTime(Date.parseExact(this.RXMap.col_eventDate this.RXMap.col_eventTime, "yyyyMMddHHmmss"))) {
    this.evtName = "Example event";
    this.evtMsg = this.RXMap.col_text1 " " this.RXMap.col_text2 " by " this.RXMap.col_userID;
    this.evtSev = 1;
    // set SEND_EVENT to true if your parsing logic worked correctly
    instance.SEND_EVENT = true;
    }
    // If you can't parse...
    else {
    rec.sendUnsupported();
    }
    return true;
    };

    --------------------


    The funny thing is that e.setObserverEventTime() did not return errors
    or false and the variable eventTime used previously seemed to be ok in
    the debugger.

    Anyways got it working. Hope this helps someone struggling with similar
    trouble.

    -Petri


    --
    petri_vesamaki
    ------------------------------------------------------------------------
    petri_vesamaki's Profile: https://forums.netiq.com/member.php?userid=379
    View this thread: https://forums.netiq.com/showthread.php?t=51166


  • Finally got it working with the help of NetIQ support. Thank you.

    So the problem was with setting the date. The debugger showed no errors
    and displayed the time correctly, but under the hood there was something
    wrong, which I don't quite understand.

    This is how it didn't work:

    Code:
    --------------------

    Record.prototype.parse = function(e){
    ...
    // The DateYYYY etc. are set from two columns in the table, nothing special about them
    var eventTime = new Date(DateYYYY,DateMM,DateDD,TimeHH,TimeMM,TimeSS);

    ...
    if (e.setObserverEventTime(DateTime)) {
    this.evtName = "Example event";
    this.msg = this.RXMap.col_text1 " " this.RXMap.col_text2 " by " this.RXMap.col_userID;
    // set SEND_EVENT to true if your parsing logic worked correctly
    instance.SEND_EVENT = true;
    }
    // If you can't parse...
    else {
    rec.sendUnsupported();
    }
    }

    --------------------


    And this is how it works:


    Code:
    --------------------

    Record.prototype.parse = function(e){

    if (e.setObserverEventTime(Date.parseExact(this.RXMap.col_eventDate this.RXMap.col_eventTime, "yyyyMMddHHmmss"))) {
    this.evtName = "Example event";
    this.evtMsg = this.RXMap.col_text1 " " this.RXMap.col_text2 " by " this.RXMap.col_userID;
    this.evtSev = 1;
    // set SEND_EVENT to true if your parsing logic worked correctly
    instance.SEND_EVENT = true;
    }
    // If you can't parse...
    else {
    rec.sendUnsupported();
    }
    return true;
    };

    --------------------


    The funny thing is that e.setObserverEventTime() did not return errors
    or false and the variable eventTime used previously seemed to be ok in
    the debugger.

    Anyways got it working. Hope this helps someone struggling with similar
    trouble.

    -Petri


    --
    petri_vesamaki
    ------------------------------------------------------------------------
    petri_vesamaki's Profile: https://forums.netiq.com/member.php?userid=379
    View this thread: https://forums.netiq.com/showthread.php?t=51166