Welcome Serena Central users! CLICK HERE
The migration of the Serena Central community is currently underway. Be sure to read THIS MESSAGE to get your new login set up to access your account.

NetIQ Operations Center - Tips for key field in Data Integrator

NetIQ Operations Center - Tips for key field in Data Integrator

The other day I was helping a customer with errors in the NOC (NetIQ Operations Center) log file.   They had built several integrations into different databases using Data Integrator (AKA: DI, BDI).   The adapter originally built worked fine but over time some errors showed up complaining with messages like: Could not create alarm with key.

A lot of applications store data into database tables and in order to bring the data into NOC, you need to have a basic understanding of the data and an idea of how you want NOC to process it.   Some applications store one row with all the data and the row has a unique identifier for the single record such as a single help desk ticket, a single change request, an alarm, etc.  Others may have multiple rows for a single item, some (many) have one piece of data in one table with additional data in other tables.  Regardless, NOC needs to have a unique identifier so that it has an understanding that when the data is read in, is NOC supposed to update an existing alarm (ie: set it from Opened to Closed) or create a new one, in order to do that, you must assign that unique identifier as a key field.

Sometimes what happens is that the "key field" is not obvious, or for your integration requirements the row number or GUID is not really what you want.  For instance, maybe the host name of the computer (ie: one column in the table) is really what you want.   So based on that, you click the keyfield next to the host name in the Data Integrator Editor.   Things are working great until one day, someone adds a new computer entry to the application but they omit entering a computer name for some reason, maybe they don't know what it will be named yet.  NOC does not like NULL values as a key and in turn, it will complain in the log file AND it may toss that record/row of data on the floor.

In order to alleviate the possibility of this happening, a good practice is to check for keyfields not being NULL.   Let's assume your "where" clause on the select statement was looking for computers in production such as PROD='YES' and maybe a particular subnet such as SUBNET like "192%".  My recommendation is to add on to that with the null check for the keyfields...

PROD = 'YES' and

SUBNET like '192%' and

hostname != nulll

 

Depending on the database vendor, there are other option for checking for NULL.  Also, remember NULL is not the same as an empty field, you may need to check for both cases.

hostname!=null and

hostname != ''

So now you say, but wait, if the hostname isn't filled I won't see the new record in NOC and I need to be made aware of these.   What you can do is create another view/tree under the Data Integrator adapter that is purposely pulling in the NULL keyfields, just use a different column/field such as the IP address, or compound other fields together, or go back to using a GUID.  You'll have to figure it out.  What ever will cause an individual row to create an item in NOC so you can see it.  Remember, the view will not be hostnames... it is empty.

You could also put a full blow 'CASE' statement on your key, if blank or null set it to "None Specified", but be prepared to provide enough data out of the other fields so it can be tracked down.  Also, if you have 100 hostnames equal to NULL, they will all collapse down to a single item in NOC if you set the key to "None Specified"... remember, it is the keyfield.

Another thing on keyfields is that sometimes you need to compound multiple things in order to really get the keyfield.   Some people just check off multiple boxes and say use field One, Two and Three as the key.  Personally, again, my opinion, is that I create an adhoc field, call it "keyfield", click the checkbox and then I construct it myself via sql.  I feel it is more obvious to the next person coming along doing edits to the definitions.

Assuming MS SQL:   Field1 + Field2 + Field3

Oracle:  concat(Field1, concat(field2, field3) )

You may even want it readable such as Field1 + ":" + Field2 + ":" + Field3

Sometimes key Fields can help you troubleshooting problems (IE: one of the fields being empty)

I'm not going to be able to answer all the scenario's, just give you some ideas on things you can do.   Hopefully you will find this blog useful.

- Tobin

DISCLAIMER:

Some content on Community Tips & Information pages is not officially supported by Micro Focus. Please refer to our Terms of Use for more detail.
Top Contributors
Version history
Revision #:
1 of 1
Last update:
‎2015-03-16 23:32
Updated by:
 
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.