Anonymous_User Absent Member.
Absent Member.
700 views

Developed Collector for Oracle DB giving error


Hi All,
We are using Sentinel Log Manager 1.2, and during development and
testing of one of our collectors for a software we are getting the
following "Collector Internal Message",


-"MaxRows parameter is too small for this database source!"

-

-"MaxRows query size too large to handle!"-

The collector is retrieving application specific audit data from
application table from Oracle database

Can anyone please specify how the MaxRows parameter can be specified,
the collector has CONNECTION_METHOD as "DATABASE" and CONNECTION_MODE as
"map"

hope I have posted it in correct forum.

Regards,

Saqib


--
saqibmabbasi
------------------------------------------------------------------------
saqibmabbasi's Profile: https://forums.netiq.com/member.php?userid=2399
View this thread: https://forums.netiq.com/showthread.php?t=42674

0 Likes
9 Replies
Anonymous_User Absent Member.
Absent Member.

Re: Developed Collector for Oracle DB giving error

saqibmabbasi,

It appears that in the past few days you have not received a response to your
posting. That concerns us, and has triggered this automated reply.

Has your problem been resolved? If not, you might try one of the following options:

- Visit http://support.novell.com and search the knowledgebase and/or check all
the other self support options and support programs available.
- You could also try posting your message again. Make sure it is posted in the
correct newsgroup. (http://forums.novell.com)

Be sure to read the forum FAQ about what to expect in the way of responses:
http://forums.novell.com/faq.php

If this is a reply to a duplicate posting, please ignore and accept our apologies
and rest assured we will issue a stern reprimand to our posting bot.

Good luck!

Your Novell Product Support Forums Team
http://forums.novell.com/

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Developed Collector for Oracle DB giving error


The MaxRows parameter is specified in the collector properties (ESM ->
right click collector -> edit -> Maximum Rows to Return.
But I guess your real problem could have something to do with not using
the %d in your sqlquery.base.
See the folowing link for more information about that:
http://tinyurl.com/9nujw6u
To be short your sqlquery.base should look something like
SELECT * FROM table WHERE key > %s and rownum < %d
%s is your offset (handeld automaticly by the connector) and %d is the
maximum number of rows.


--
jcvader1
------------------------------------------------------------------------
jcvader1's Profile: https://forums.netiq.com/member.php?userid=502
View this thread: https://forums.netiq.com/showthread.php?t=42674

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Developed Collector for Oracle DB giving error


Hi jcvader1,
Thanks for the reply, i have used both "%d" and "%s", initially I was
using the "%s" by timestamp column name but it was returning error
related to picture format of the timestamp, so i had changed to column
type to number,b but now it is giving this error.

I know that parameter "Maximum Rows to Return" is the actual field, but
even if i change the value it returns error, I opened the code of
Connector and found that it is checking the MaxRows with 5000 in case of
bulk. But it is also multiplying the value with 2.

Suppose if the initial value was set to 2000 then it will perform the
following iterations.
>
> 1. 2000 = 2000 checks with 5000 - will give the message (in
> Yellow) - "MaxRows parameter is too small for this database source!"
> 2. 2000 * 2 = 4000 checks with 5000 - will give the message (in
> Yellow) - "MaxRows parameter is too small for this database source!"
> 3. 4000 * 2 = 6000 checks with 5000 - will give the message (in Red)
> - "MaxRows query size too large to handle!"
>


After the third iteration the Collector-Connector-Event Source stops.

My question is how do I change it to not use the bulk.

Regards,
Saqib.


--
saqibmabbasi
------------------------------------------------------------------------
saqibmabbasi's Profile: https://forums.netiq.com/member.php?userid=2399
View this thread: https://forums.netiq.com/showthread.php?t=42674

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Developed Collector for Oracle DB giving error


I can't follow you at all.
Lets start with what collector you are using, is it a self developed
one or a default collector (if so which one)?
You changed the offset column to a number, are you sure this number is
unique?
Did you change the select statement or the actual column in the
database?
I googled the error-code and if I understand correctly you get the
MaxRows parameter is too small for this database source message if all
max rows are retreived and the offset hasn't changed.
Did you put the parser correct in your initialization code?
For instance:
// Example code to define a parser that can determine the latest
offset from new records
// Use this for DATABASE sources (see SQLQuery class)
this.PARSER.getOffsetData = function(input){
// parser code, like "return input.RXMap.col_AutoID"
return input.RXMap.col_ROWKEY }
conn.addParser(this.PARSER.getOffsetData);

The important bit is the: return input.<your offset column>

without this the offset isn't changed and you get the maxrows error.

Hope this helps.
Anco


--
jcvader1
------------------------------------------------------------------------
jcvader1's Profile: https://forums.netiq.com/member.php?userid=502
View this thread: https://forums.netiq.com/showthread.php?t=42674

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Developed Collector for Oracle DB giving error


Hi jcvader1,

Got that problem fixed. I'm creating a new Collector using the Sentinel
SDK for a off-the-list system, I was using the Database connector to
catch events from application specific tables in Oracle database.

For other peoples that might encounter the same problem...

There are two Offsets. The primary "Offset" and the "SubOffset". I had
changed the return criteria for Offset but not changed the return
criteria for SubOffset, which was still using the Oracle specific
criteria. After fixing this the problem was resolved.

Now the error is occurring during parsing, therefore checking it out.

By the way, I do have another question, can we use two separate queries
to collect the data. If so, how?

Regards,
Saqib.


--
saqibmabbasi
------------------------------------------------------------------------
saqibmabbasi's Profile: https://forums.netiq.com/member.php?userid=2399
View this thread: https://forums.netiq.com/showthread.php?t=42674

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Developed Collector for Oracle DB giving error


Hi Saqib,

Glad you got that fixed.
If you don't use the Suboffset just delete the code for it.

To answer your other question, yes you can, you will have to create two
instances of the collector to do this.
You can choose which query in the "edit"-part of the collector (near
the Maximum rows to return).
As far as I am aware there is no documentation yet.
I had to look back some sdk beta calls a few times to figure it out.
Make sure you are using the latest SDK
I'm not quite sure about everything but if you want 2 queries here is
what you should do:
Open 2011.1/parameters/template.pml and add the following line at the
bottom:
Query_Variant
now create a sqlquery1.base in 2011.1 with the first query
create a sqlquery2.base in 2011.1 with the second query
if you build your collector now a parameter is added: Query Variant
Support
Here you can choose between Variant1 (sqlquery1.base) and Variant2
(sqlquery2.base)
In the file 2011.1/parameters/query_variant.xml
(query_variant_sample.xml is an example) you can put a name of your own
but I never got that to work for some reason.

If I'm correct that should be it.

Regards,
Anco


--
jcvader1
------------------------------------------------------------------------
jcvader1's Profile: https://forums.netiq.com/member.php?userid=502
View this thread: https://forums.netiq.com/showthread.php?t=42674

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Developed Collector for Oracle DB giving error


Hi Anco,
I think I will hang-on to the suboffset and use it 🙂

As for the Query Variant thanks will check it out and let you know if
it works.

Basically there are two schema's one for administration and the other
for actual application data, both have separate audit table in their
schema, the structure is same therefore the collector logic is same, so
was planning to use the query variant and then add separate collector to
the Log Manager box, and define the specific query variant.

This ought to work.

Regards,
Saqib


--
saqibmabbasi
------------------------------------------------------------------------
saqibmabbasi's Profile: https://forums.netiq.com/member.php?userid=2399
View this thread: https://forums.netiq.com/showthread.php?t=42674

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Developed Collector for Oracle DB giving error


Hi guys,
Referring to the same collector, In case, Oracle table has a field with
data type XMLType, how can we pick the XML from the field and then parse
it.

I have tried following two methods:

> select xmlfield from tablename;


> select xmlfield.getCLOBval from tablename;


But in both the cases when trying to write the xml (without parsing) to
message it is giving "undefined".
Is there any other method.

Regards,

Saqib


--
saqibmabbasi
------------------------------------------------------------------------
saqibmabbasi's Profile: https://forums.netiq.com/member.php?userid=2399
View this thread: https://forums.netiq.com/showthread.php?t=42674

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Developed Collector for Oracle DB giving error


Hi guys,

Finally got it working by using the second option, somehow possibly I
was making some mistake.

Regards,
Saqib.


--
saqibmabbasi
------------------------------------------------------------------------
saqibmabbasi's Profile: https://forums.netiq.com/member.php?userid=2399
View this thread: https://forums.netiq.com/showthread.php?t=42674

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.