Highlighted
Absent Member.
Absent Member.
690 views

customizing connector.js?


Hello,

I try to change the DB-Connector for our DB2 connector and need to
change some timings in connector.js.
I added a copy from the SDK_2014 to the connector source directory.
But after compiling the compiler replaces the file from the common-SDK
directory. Adding it into files.include does not work.
What is the best way to include some changed files from the common-dir
of the SDK into the conenctor zip-file?

Torsten


--
tfechner
------------------------------------------------------------------------
tfechner's Profile: https://forums.netiq.com/member.php?userid=8929
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
16 Replies
Highlighted
Knowledge Partner
Knowledge Partner

I've rebuilt plugins in the past by just extracting it, modifying things,
then putting it back. I do not see why that would not work in your case.

Have you tried something and had issues? Also, could you explain your
timing issues so the plugin SDK could be improved?


--
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below...
0 Likes
Highlighted
Absent Member.
Absent Member.


tfechner;263890 Wrote:
> Hello,
>
> I try to change the DB-Connector for our DB2 connector and need to
> change some timings in connector.js.
> I added a copy from the SDK_2014 to the connector source directory.
> But after compiling the compiler replaces the file from the common-SDK
> directory. Adding it into files.include does not work.
> What is the best way to include some changed files from the common-dir
> of the SDK into the conenctor zip-file?
>
> Torsten


Hii Torsten,

My general answer to customizing connector.js is "Don't". ALthough
we're still working through plans, expect connector.js not to exist in a
coming release of the SDK.

What is the issue you are experiencing that makes you want to adjust
connector.js? I saw a question on one of our internal lists about this
coming from support, do you have an SR open on this?


--
brandon.langley
------------------------------------------------------------------------
brandon.langley's Profile: https://forums.netiq.com/member.php?userid=350
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
Highlighted
Absent Member.
Absent Member.


Hi,

in the old SDK (2011) I used the connector.js local in my plugin
directory to change the default behaviour.

I would like to change this (an SR was opened, too)
I have a connector using database-connection.
At
this.read = function() {
there is a timing/schedule function. Default ther is a waiting time
from 5 secs. I changed this zu 1800 secs to have only a SQL send every
30min.
5 sec would kill our DB.

for (var i=0; i < this.retries; i++) {
// short retry to ensure that we check if we should keep running
*record = getData(10); *


A little bit further on at
if (record.CONNECTION_METHOD == "DATABASE") { /
.....

if (instance.QUERIES[record.s_RV24].queryDelay > 0 &&
instance.QUERIES[record.s_RV24].queryDelay < 33) {
instance.QUERIES[record.s_RV24].queryDelay=instance.QUERIES[record.s_RV24].queryDelay
* 2;
} else {
instance.QUERIES[record.s_RV24].queryDelay = 1;
}
instance.QUERIES[record.s_RV24].queryScheduled =
DateTime.getMilliseconds(DateTime.getDateTime()) +
(instance.QUERIES[record.s_RV24].queryDelay * 2);

This is my changing point:
Do not cycle to 33. Do only two cyle, but do next schedule not at
delay*2 miliseconds, better do it at.. let's say 10 min later:

instance.QUERIES[record.s_RV24].queryScheduled =
DateTime.getMilliseconds(DateTime.getDateTime()) + 600000;


Same at:
// Special suboffset handling
if ( instance.QUERIES[record.s_RV24].SubOffsets == true ) {
......


But I am wondering why my local adapted files in the plugin directory
are not copied to the zip-file.

Torsten


--
tfechner
------------------------------------------------------------------------
tfechner's Profile: https://forums.netiq.com/member.php?userid=8929
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
Highlighted
Absent Member.
Absent Member.


tfechner;263935 Wrote:
> Hi,
>
> in the old SDK (2011) I used the connector.js local in my plugin
> directory to change the default behaviour.
>
> I would like to change this (an SR was opened, too)
> I have a connector using database-connection.
> At
> this.read = function() {
> there is a timing/schedule function. Default ther is a waiting time
> from 5 secs. I changed this zu 1800 secs to have only a SQL send every
> 30min.
> 5 sec would kill our DB.
>
> for (var i=0; i < this.retries; i++) {
> // short retry to ensure that we check if we should keep running
> *record = getData(10); *
>
>
> A little bit further on at
> if (record.CONNECTION_METHOD == "DATABASE") { /
> ....
>
> if (instance.QUERIES[record.s_RV24].queryDelay > 0 &&
> instance.QUERIES[record.s_RV24].queryDelay < 33) {
> instance.QUERIES[record.s_RV24].queryDelay=instance.QUERIES[record.s_RV24].queryDelay
> * 2;
> } else {
> instance.QUERIES[record.s_RV24].queryDelay = 1;
> }
> instance.QUERIES[record.s_RV24].queryScheduled =
> DateTime.getMilliseconds(DateTime.getDateTime()) +
> (instance.QUERIES[record.s_RV24].queryDelay * 2);
>
> This is my changing point:
> Do not cycle to 33. Do only two cyle, but do next schedule not at
> delay*2 miliseconds, better do it at.. let's say 10 min later:
>
> instance.QUERIES[record.s_RV24].queryScheduled =
> DateTime.getMilliseconds(DateTime.getDateTime()) + 600000;
>
>
> Same at:
> // Special suboffset handling
> if ( instance.QUERIES[record.s_RV24].SubOffsets == true ) {
> .....
>
>
> But I am wondering why my local adapted files in the plugin directory
> are not copied to the zip-file.
>
> Torsten


I'd assume because you are modifying them on the Sentinel disk rather
than modifying them in the zip file and re-importing. Now, I think
you're the SR I'm thinking of, and I'm reviewing the information I
provided support, who btw opened Bug 944464, but (and I don't know the
entire story), the SR appears to have been closed so the issue was
marked solution provided - it doesn't sound like that's the case, and I
was certainly willing to discuss this issue further - so maybe there was
a disconnect there, or maybe you're not the same guy 🙂

queryDelay is a value that is set on the event source (which we
represent by the 'rv24' value). I noticed that this value is capped at
100, and you probably need us to change that to support a fairly large
value - I saw one request for 6 hours, so it's not unreasonable to
support arbitrarily large values here.

We'd probably also need to change the collector framework (and update
the specific collectors you use) to remove the automatically increasing
adjustment once the retry timeout is over a certain size (basically I
think if it's > 30 seconds, it's fixed at the queryDelay value).

If you think this adjustment would work, I would re-open the SR,
reference this forum thread, and let's get you fixed - and if there are
fine-tuning that we need to do to make this work for you, I think it's
worth considering.

Lastly, if the performance impact of a single query when there is no
data to retrieve is causing undue stress on your database, that's
probably an issue with the collector outright that we need to look into.
As a general rule, we don't expect a no-data-returned query to be
something that should be so much stress it can't be run every 30
seconds.


--
brandon.langley
------------------------------------------------------------------------
brandon.langley's Profile: https://forums.netiq.com/member.php?userid=350
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
Highlighted
Absent Member.
Absent Member.


HI,

yes - I was the one who initiated this SR.

Background:
On IBM-DB2 the creation of audit-data consumes too much performance to
be run all the time. So we create this audit data only every 6 hours:
export the data from the tables, prepair them and import them into the
audit-tables - as a normal audit way on DB2. This process needs some CPU
cycles and so it runs only every 6 hours.

The connector connects every 5 seconds to the DB... some seconds more or
less depending on the cyle. Every access is logged for audit....
And why connecting so often if there is no data in there...
So I requested a SR but as I can remember the solution was: there is no
way to change and works as designed (a normal answer) . So I hacked into
the code and expanded the timer from 5 secs to 1800 secs. Quick and
dirty. Works.
But changing the scheduled time is a better programming style.
Therefore I need to change the connector.js.

But in gerneral:
Using the 2011-SDK I put some files from the SDK into the plugin
directory and changed them. At compling time all my local changed files
were copied into the plugin (Logfile: there was only one file beeing
copied)

Now in SDK2014 (from the logfile in eclipse)
_copy-plugin: my own files are copied to the build-dir
_copyAPI: now the 20 sdk files are copied into the build dir,
overwriting the previous copied files!
...building html docu.
done.


Is there a cfg- file for the SDk to change this mechanism?

Torsten


--
tfechner
------------------------------------------------------------------------
tfechner's Profile: https://forums.netiq.com/member.php?userid=8929
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
Highlighted
Absent Member.
Absent Member.


tfechner;263960 Wrote:
> HI,
>
> yes - I was the one who initiated this SR.
>
> Background:
> On IBM-DB2 the creation of audit-data consumes too much performance to
> be run all the time. So we create this audit data only every 6 hours:
> export the data from the tables, prepair them and import them into the
> audit-tables - as a normal audit way on DB2. This process needs some CPU
> cycles and so it runs only every 6 hours.
>
> The connector connects every 5 seconds to the DB... some seconds more or
> less depending on the cyle. Every access is logged for audit....
> And why connecting so often if there is no data in there...
> So I requested a SR but as I can remember the solution was: there is no
> way to change and works as designed (a normal answer) . So I hacked into
> the code and expanded the timer from 5 secs to 1800 secs. Quick and
> dirty. Works.
> But changing the scheduled time is a better programming style.
> Therefore I need to change the connector.js.
>
> But in gerneral:
> Using the 2011-SDK I put some files from the SDK into the plugin
> directory and changed them. At compling time all my local changed files
> were copied into the plugin (Logfile: there was only one file beeing
> copied)
>
> Now in SDK2014 (from the logfile in eclipse)
> _copy-plugin: my own files are copied to the build-dir
> _copyAPI: now the 20 sdk files are copied into the build dir,
> overwriting the previous copied files!
> ..building html docu.
> done.
>
>
> Is there a cfg- file for the SDk to change this mechanism?
>
> Torsten


I think the SR needs to be re-opened. What you're asking here is not
what was communicated to us at the time, but what you are looking to do
above should be something pretty addressable. You could always modify
the local SDK version, but if you are doing this for DB2 (a supported
collector), I would prefer that we open a SR against that specific
collector, referencing this forum post, and we should be able to get
this adjusted to provide a fixed query interval.


--
brandon.langley
------------------------------------------------------------------------
brandon.langley's Profile: https://forums.netiq.com/member.php?userid=350
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
Highlighted
Absent Member.
Absent Member.


we could reopen or recreate a SR.
But as our DB2-Connectors are pretty much customized we need the
solution from the developer to add the changes to our version.


--
tfechner
------------------------------------------------------------------------
tfechner's Profile: https://forums.netiq.com/member.php?userid=8929
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
Highlighted
Absent Member.
Absent Member.


...mh... strange..
I changed these in connector.js:

Code:
--------------------
...
this.read = function() {
var record = null;
try {
if (instance.CONFIG.params.Conn_Retry != "none") {
if (instance.CONFIG.params.Conn_Retry == "fixed") {
record = getData(instance.CONFIG.params.RX_Timeout);
} else {

// short retry to ensure that we check if we should keep running
record = getData(13); //<-- specal value to recognize in tcpdump
// Check if the Collector was stopped

}
}
....
if (record.CONNECTION_METHOD == "DATABASE")
.....
instance.QUERIES[record.s_RV24].queryScheduled = DateTime.getMilliseconds(DateTime.getDateTime()) + 17000; //<- 17sec wait
...
--------------------

No I started the connector and expected to see a query each 17 seconds
to the db:

Code:
--------------------

# tcpdump -s 10 host XXXX.
16:32:46.279271 IP [|ip]
16:32:46.279293 IP [|ip]
16:32:52.251792 IP [|ip]
16:32:52.253179 IP [|ip]
16:32:52.253203 IP [|ip]
16:32:52.253244 IP [|ip]
16:32:52.253263 IP [|ip]
16:32:52.254614 IP [|ip]
16:32:52.255766 IP [|ip]
16:32:52.293128 IP [|ip]

16:33:12.298905 IP [|ip]
16:33:12.298926 IP [|ip]

16:33:18.269218 IP [|ip]
16:33:18.270586 IP [|ip]
16:33:18.270601 IP [|ip]
16:33:18.270714 IP [|ip]
16:33:18.270735 IP [|ip]
16:33:18.272119 IP [|ip]
16:33:18.272998 IP [|ip]
16:33:18.313139 IP [|ip]

16:33:38.318938 IP [|ip]
16:33:38.318963 IP [|ip]

16:33:44.286674 IP [|ip]
16:33:44.287788 IP [|ip]
16:33:44.287803 IP [|ip]
16:33:44.287928 IP [|ip]
16:33:44.287950 IP [|ip]
16:33:44.289239 IP [|ip]
16:33:44.289915 IP [|ip]
16:33:44.329137 IP [|ip]

16:34:04.339328 IP [|ip]
16:34:04.339350 IP [|ip]

16:34:10.306694 IP [|ip]
16:34:10.308393 IP [|ip]
16:34:10.308425 IP [|ip]
16:34:10.308533 IP [|ip]
16:34:10.308548 IP [|ip]
16:34:10.309976 IP [|ip]
16:34:10.310817 IP [|ip]
16:34:10.349135 IP [|ip]

16:34:14.720120 IP [|ip]
16:34:14.721412 IP [|ip]
16:34:14.721432 IP [|ip]

--------------------


Every 20 seconds a query is sent to the DB.
It seems that the "queryScheduled =
DateTime.getMilliseconds(DateTime.getDateTime()) + 17000" does not
work?

or where can I find the default wait time of 20sec?

btw: one can configure a receiving time out in the ESM in the collector
gui: it is set to 35 secs!
(and ... only tis connector is active on this test-sentinel)


--
tfechner
------------------------------------------------------------------------
tfechner's Profile: https://forums.netiq.com/member.php?userid=8929
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
Highlighted
Absent Member.
Absent Member.


Correction:
deep analysis with a whireshark shows a sql-sent , then some keepalives,
NO shutdown of the db connection and after the configured time the next
sql statement is sent.
So all is working fine.


--
tfechner
------------------------------------------------------------------------
tfechner's Profile: https://forums.netiq.com/member.php?userid=8929
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
Highlighted
Absent Member.
Absent Member.


tfechner;264024 Wrote:
> Correction:
> deep analysis with a whireshark shows a sql-sent , then some keepalives,
> NO shutdown of the db connection and after the configured time the next
> sql statement is sent.
> So all is working fine.


Hi there,

So you're saying that the bit that happens every 20s is just a keepalive
to keep the connection active, and doesn't cause and DB activity?

And then the actual query against the DB now happens on your desired
interval?


--
DCorlette
------------------------------------------------------------------------
DCorlette's Profile: https://forums.netiq.com/member.php?userid=323
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
Highlighted
Absent Member.
Absent Member.


DCorlette;264362 Wrote:
> Hi there,
>
> So you're saying that the bit that happens every 20s is just a keepalive
> to keep the connection active, and doesn't cause and DB activity?
>
> And then the actual query against the DB now happens on your desired
> interval?


HI,
yes. the tcpdump shows that only a short packet is sent to the DB, the
ascii-decode shows no readable text.
I seem to me that the tcp connection to the db is opend, a query sent
and after this the connection is beeing kept alive. A next query is
scheduled to the given time ( at my connector 30 min later).


--
tfechner
------------------------------------------------------------------------
tfechner's Profile: https://forums.netiq.com/member.php?userid=8929
View this thread: https://forums.netiq.com/showthread.php?t=55085

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.