Highlighted
Super Contributor.
Super Contributor.
1298 views

UCMDB-UCMDB Population Integration

Hi All,

 

We have  two UCMDB 10.20 servers: UCMDB-1 and UCMDB-2.

UCMDB-1 is the GlobalID generator act as a CMS.
UCMDB-2 (non-GlobalID generator) run discover jobs.

I created Integration Point based on UCMDB 10.x adapter. Created and deployed to UCMDB-2 a simple TQL Node->Databases.

 

After run population job I see in error.log ErrorCode [63010] Identification map total size is larger than the defined fuse.{6000000}

Spoiler
2015-04-27 13:16:40,703  ERROR  [Process Results Thread-DS_UCMDB_FD_DB] (AbstractDataInUpdateOperation.java:271) - !!!FAILED!!![Operation #148503361 Name=DataInAddOrUpdateData]
com.hp.ucmdb.reconciliation.datain.operation.DataInOperationException: [ErrorCode [63010] Identification map total size is larger than the defined fuse.{6000000}]
Error while trying to [addOrUpdate] on level [1]! Error appending ID $TempObjectaa19a16019b839d5a126e730864913cd to identification map. Total size of map is larger than the limit 6000000 which defined in reconciliation.identification.map.fuse

 

I set Population Chunk Size to 20  and reboot DFP  - it did not help.

I set appilog.agent.local.maxTaskResultSize = 100 in DataFlowProbe.properties - it did not help.

 

How to correctly configure the UCMDB to UCMDB integration?

Best Regards, Sergy.
0 Likes
8 Replies
Highlighted
New Member.

Re: UCMDB-UCMDB Population Integration

Not sure I understand exactly what you are trying to accomplish.

GlobalIds are generallly generated during UD discovery so I don't understand how you are popluationg UCMDB-1.

 

You may want to try to limit the data set returned by the TQL or even break it up some.

Remimber the fuse is based on number of CIs fetched, which includes relationships too.

 

Not sure which fuse  you are referring to but the message provided implies it's on the UCMDB server not the DFM.

Fuses are used to prevent the application from being over run so take care when altering them.

0 Likes
Highlighted
Super Contributor.
Super Contributor.

Re: UCMDB-UCMDB Population Integration

Thanks for Your answer.


>Not sure I understand exactly what you are trying to accomplish.

I try to populate CIs from UCMDB-2 to another UCMDB-1. Population job runs on UCMDB-1.

>GlobalIds are generallly generated during UD discovery so I don't understand how you are popluationg UCMDB-1

GlobalIDs generates by UCMDB with GlobalID generator role(in my case UCMDB-1), UCMDB-2 doesn't has GlobalIDs (because it sets as non-GlobalID generator). Population job running on UCMDB-1 and push back GlobalIDs. But my question doesn't about GlobalID.

>Remimber the fuse is based on number of CIs fetched, which includes relationships too.

Exactly. I know.

>You may want to try to limit the data set returned by the TQL or even break it up some.

Yes, How can I to limit the data set returned by the TQL? I know only one method - add conditions to TQL.

 

I created new TQL (Node->Database) with specific attributes which returns Nodes with Database. This TQL returns  80 Nodes. But the error still exist.
Then I add more stringent conditions to TQL and it returns 20 nodes with databases.

With 20 nodes population works fine but it isn't a good idea have many TQLs for Databases.


Is there an easy way to break the received data into small portions within population job?

Thank You.

 

PS: In the "Data Flow Management Guide" I find only general information about Integration multiple CMDBs (page 368).

 

Best Regards, Sergy.
0 Likes
Highlighted
Absent Member.. Absent Member..
Absent Member..

Re: UCMDB-UCMDB Population Integration

I am trying to achieve the same thing.  We have a primary uCMDB instance to use as CMS and a secondary uCMDB that is used for discovery.  Currently all data is in the secondary uCMDB.  The primary is a new instance and we are currently testing integration.  We want to use an integration to pull the data from this secondary instance, with all of the data from discovery, into the clean primary instance. 

 

I have created this integration and it runs fine on a small scale.  But I have to limit the query results for each job and the UI warns when more than 10,000 items are returned, which can be only a few thousand hosts and their relationships.  How can we ever integrate 2 instances like this without having to create hundreds or thousands of queries in the integration?  I have tried changing the Element Layout in the query to return only required items, and that does improve things, but I have about 1 million CIs I need to move over.  I'm interested to know if anyone has had any experience with this and been successful trying to integrate 2 uCMDB instances this way.

0 Likes
Highlighted
Super Contributor.
Super Contributor.

Re: UCMDB-UCMDB Population Integration

Hi.

 

I solved my problem as follows:

 

1. I deleted or Cis from UCMDB-2 (discovery)

2. Reran discovery jobs
3. Created new TQLs. Also I added ElementLayout specific attributes for CITs in TQL.

4. Run Population integration from UCMDB-1 (CMS) with Puchback IDs option.

 

PS: Did You set your second UCMDB(with run discovery) as non global ID generator?

Best Regards, Sergy.
0 Likes
Highlighted
Absent Member.. Absent Member..
Absent Member..

Re: UCMDB-UCMDB Population Integration

Great news, were you able to integrate the 2 with just the one query shown in the screen shot, or did you have to break it down into many query jobs?

0 Likes
Highlighted
Super Contributor.
Super Contributor.

Re: UCMDB-UCMDB Population Integration

Yes, I have TQLs to populate Databases, Web Servers and ApplicationServers and different jobs.

But I will try to create a more general TQL (Node->RuanningSoftware->ApplicationResource) and run Population Job in a week.

 

Best Regards, Sergy.
0 Likes
Highlighted
Absent Member.. Absent Member..
Absent Member..

Re: UCMDB-UCMDB Population Integration

If you are able, please share any information you can about the queries you are using with the integration.  We have tried simple queries like pulling all nodes, but we get errors about too much info returned.

 

Thanks,

Jake

0 Likes
Highlighted
Valued Contributor.. Valued Contributor..
Valued Contributor..

Re: UCMDB-UCMDB Population Integration

Hi all, bringing up this old thread, hoping to get help for a newbie!   We are trying to accomplish the same thing - UCMDB1 is for discovery and is set as a non-global ID generator.  UCMDB2 is our CMS and is set as the global ID generator.  We are going to run population jobs on UCMDB2 to pull in the data from UCMDB1 and push back the global IDs.  I am struggling with the TQLs and support said I should use RTSMtoCMSSync as an example starting point.  We are on version 10.32 and I understand it handles larger volumes better than prior versions.  Has anyone had luck with using just a few TQLs for this integration?  I was considering one TQL for Nodes, another for Installed Software and a third for Software Utilization.  We will likely have around 90K nodes and 5M installed software CIs so I'm guessing I'll have to break installed software and software utilization into smaller chunks (but I was hoping to have one TQL for the nodes).  Any input is appreciated and if you can share some example TQLs that would be awesome too!

 

Thanks,

Dee

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.