Highlighted
Respected Contributor.
Respected Contributor.
834 views

uCMDB-SNOW Integration via Push Adapter

I receive below error always while pushing CIs to snow. Connection is fine. But i can insert entry in snow via SOAPUI with same wsdl link and credentials. Let me know for any idea?

 

2019-07-29 11:16:46,979 TRACE [AdHoc:AD_HOC_TASK_PATTERN_ID-1129-1564391806685] - Adding u_imp_ucmdb_server RTN to the u_imp_ucmdb_server cache. ResultTreeNode. QueryNodeName: u_imp_ucmdb_server
ExternalCiId:***{UCMDB%0Ant%0A1%0Ainternal_id%3DSTRING%3D4a33dfc5e50e7497835c27875696ff7d%0A}***
class name: nt
isCmdbId: true
cmdbId: 4a33dfc5e50e7497835c27875696ff7d
Properties:
internal_id=4a33dfc5e50e7497835c27875696ff7d
Properties:
u_name = testci
u_rwe_correlationid = 4a33dfc5e50e7497835c27875696ff7d,

2019-07-29 11:16:46,979 ERROR [AdHoc:AD_HOC_TASK_PATTERN_ID-1129-1564391806685] - Ignoring u_imp_ucmdb_server RTN tree because of invalid key. ExternalCiId:***{UCMDB%0Ant%0A1%0Ainternal_id%3DSTRING%3D4a33dfc5e50e7497835c27875696ff7d%0A}***
class name: nt
isCmdbId: true
cmdbId: 4a33dfc5e50e7497835c27875696ff7d
Properties:
internal_id=4a33dfc5e50e7497835c27875696ff7d

2019-07-29 11:16:46,979 WARN [AdHoc:AD_HOC_TASK_PATTERN_ID-1129-1564391806685] - Unable to create global CI ID for RTN. Using the original ExternalCiId: ExternalCiId:***{UCMDB%0Ant%0A1%0Ainternal_id%3DSTRING%3D4a33dfc5e50e7497835c27875696ff7d%0A}***
class name: nt
isCmdbId: true
cmdbId: 4a33dfc5e50e7497835c27875696ff7d
Properties:
internal_id=4a33dfc5e50e7497835c27875696ff7d

java.lang.NullPointerException
at com.hpe.ucmdb.adapters.util.UcmdbUtils.createGlobalExternalCiId(UcmdbUtils.java:70)
at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.reportAdditionStatus(ServiceNowPusher.java:623)
at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.addOrUpdateRtn(ServiceNowPusher.java:229)
at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.pushTreeNodes(ServiceNowPusher.java:203)
at com.hp.ucmdb.adapters.snow.ServiceNowGenericAdapter.pushTreeNodes(ServiceNowGenericAdapter.java:179)
at com.hp.ucmdb.adapters.GenericAdapter.pushToRemoteDataSource(GenericAdapter.java:575)
at com.hp.ucmdb.adapters.GenericAdapter.updateData(GenericAdapter.java:408)
at com.hp.ucmdb.dataAccess.manager.DirectBasicDataAdapterWrapper.updateData(DirectBasicDataA

0 Likes
23 Replies
Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

Hi @RahulGupta ,

In the last year I have extensively worked on the ServiceNow adapter and I can give you some outline of how to debug it:

1. Make sure you are using REST and not SOAP. It is far faster. In adapter.properties set:

connector.class=com.hpe.ucmdb.adapters.snow.connector.ServiceNowRestConnector
# connector.class=com.hpe.ucmdb.adapters.snow.connector.ServiceNowSoapConnector
push.connector.class=com.hpe.ucmdb.adapters.snow.connector.ServiceNowRestConnector

 

2. Enable debugging of servicenow adapter. This will show you the REST queries sent to ServiceNow and what has been received as a response. Ammend fcmdb.properties in DataFlowProbe\conf\log with:

 

#### ServiceNow Custom Configuration

log4j.category.servicenow.generic.adapter=TRACE,servicenow.generic.adapter.appender
log4j.appender.servicenow.generic.adapter.appender=com.mercury.topaz.cmdb.shared.base.log.BetterRollingFileAppender
log4j.appender.servicenow.generic.adapter.appender.File=${logs.dir}/servicenow.generic.adapter.log
log4j.appender.servicenow.generic.adapter.appender.MaxFileSize=10240KB
log4j.appender.servicenow.generic.adapter.appender.MaxBackupIndex=${def.files.backup.count}
log4j.appender.servicenow.generic.adapter.appender.layout=org.apache.log4j.PatternLayout
log4j.appender.servicenow.generic.adapter.appender.layout.ConversionPattern=${msg.layout}

 

3. Check the import set and transform map history logs in ServiceNow. If the data is properly sent there, you will be able to see what is the reason for rejection.

 

Regards,

Petko Popadiyski

Freelance Microfocus CMS UCMDB Consulting

Likes are appreciated!
Highlighted
Respected Contributor.
Respected Contributor.

Hello, Thanks for your reply. I have already done first and second step. I cannot perform third step, as we don't have access to snow. Moreover, data is not reaching to snow staging tables and that's confirmed.

Let me know for any suggestions please.

 

Regards,

Rahul

Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

if you have done 1 and 2, you have logs with the requests to ServiceNow. what are the request payloads and what are the responses?

Likes are appreciated!
0 Likes
Highlighted
Honored Contributor.
Honored Contributor.

Hello,

I'm facing the same issue when trying to push into a SNOW instance.

The TQL is fairly simple, and i've reduced the target_mapping fields to the strict minimum (global_id -> correlation_id and name -> name), but with no luck.

I've also activated traces on the probe and it seems that no call to SNOW is even done, so it's probably an issue on CMDB side, here is the output :

2020-07-17 17:17:25,157  TRACE  [AdHoc:AD_HOC_TASK_PATTERN_ID-688-1594999044252] - Adding u_imp_microfocus_server RTN to the u_imp_microfocus_server cache.     ResultTreeNode. QueryNodeName: u_imp_microfocus_server
        ExternalCiId:***{UCMDB%0Aunix%0A1%0Ainternal_id%3DSTRING%3D450339ff03f73f448ee46f66ad1c9fd6%0A}***
            class name: unix
            isCmdbId: true
            cmdbId: 450339ff03f73f448ee46f66ad1c9fd6
            Properties:
                internal_id=450339ff03f73f448ee46f66ad1c9fd6
        Properties:
            u_correlation_id = 406bb305cb7f8d89a772984d3058cc9d
            u_name = cms2lnx0002, 

2020-07-17 17:17:25,172  ERROR  [AdHoc:AD_HOC_TASK_PATTERN_ID-688-1594999044252] - Ignoring u_imp_microfocus_server RTN tree because of invalid key.    ExternalCiId:***{UCMDB%0Aunix%0A1%0Ainternal_id%3DSTRING%3D450339ff03f73f448ee46f66ad1c9fd6%0A}***
        class name: unix
        isCmdbId: true
        cmdbId: 450339ff03f73f448ee46f66ad1c9fd6
        Properties:
            internal_id=450339ff03f73f448ee46f66ad1c9fd6

2020-07-17 17:17:25,172  WARN   [AdHoc:AD_HOC_TASK_PATTERN_ID-688-1594999044252] - Unable to create global CI ID for RTN. Using the original ExternalCiId: ExternalCiId:***{UCMDB%0Aunix%0A1%0Ainternal_id%3DSTRING%3D450339ff03f73f448ee46f66ad1c9fd6%0A}***
    class name: unix
    isCmdbId: true
    cmdbId: 450339ff03f73f448ee46f66ad1c9fd6
    Properties:
        internal_id=450339ff03f73f448ee46f66ad1c9fd6

java.lang.NullPointerException
	at com.hpe.ucmdb.adapters.util.UcmdbUtils.createGlobalExternalCiId(UcmdbUtils.java:70)
	at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.reportAdditionStatus(ServiceNowPusher.java:535)
	at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.addOrUpdateRtn(ServiceNowPusher.java:223)
	at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.pushTreeNodes(ServiceNowPusher.java:199)
	at com.hp.ucmdb.adapters.snow.ServiceNowGenericAdapter.pushTreeNodes(ServiceNowGenericAdapter.java:178)
	at com.hp.ucmdb.adapters.GenericAdapter.pushToRemoteDataSource(GenericAdapter.java:817)
	at com.hp.ucmdb.adapters.GenericAdapter.processMapping(GenericAdapter.java:554)
	at com.hp.ucmdb.adapters.GenericAdapter.updateData(GenericAdapter.java:526)
	at com.hp.ucmdb.dataAccess.manager.DirectBasicDataAdapterWrapper.updateData(DirectBasicDataAdapterWrapper.java:434)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.hp.ucmdb.discovery.probe.processor.GenericPushProbeRequestProcessor.processRequest(GenericPushProbeRequestProcessor.java:36)
	at com.hp.ucmdb.discovery.probe.processor.GenericPushProbeRequestProcessor.processRequest(GenericPushProbeRequestProcessor.java:13)
	at com.hp.ucmdb.discovery.probe.processor.AbstractProbeProcessor.process(AbstractProbeProcessor.java:56)
	at com.hp.ucmdb.discovery.probe.processor.AbstractProbeProcessor.process(AbstractProbeProcessor.java:19)
	at com.hp.ucmdb.discovery.probe.agents.probemgr.adhoctasks.AdHocProbeRequestOperation.performAction(AdHocProbeRequestOperation.java:64)
	at com.hp.ucmdb.discovery.probe.agents.probemgr.taskdispatcher.AdHocTaskDispatcher.dispatchTask(AdHocTaskDispatcher.java:70)
	at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
	at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
	at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
	at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
	at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
	at javax.management.StandardMBean.invoke(StandardMBean.java:405)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
	at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:298)
	at org.springframework.jmx.access.MBeanClientInterceptor.doInvoke(MBeanClientInterceptor.java:417)
	at org.springframework.jmx.access.MBeanClientInterceptor.invoke(MBeanClientInterceptor.java:366)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy66.dispatchTask(Unknown Source)
	at com.hp.ucmdb.discovery.probe.agents.probegw.managementtasks.adhoctasks.AdhocThread.run(AdhocThread.java:54)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
	at java.lang.Thread.run(Thread.java:748)

 

Of course i've triple checked the inbound service name and properties on SNOW side, and there is nothing in import sets logs since no call is done.

Any clue or thing i can do to debug this ?

Thanks,
Best regards,
Yann Pingot

0 Likes
Highlighted
Respected Contributor.
Respected Contributor.

Please check first, if your xml file have correlation_id attribute for snow.  Are you using staging table to push data?

Let me know, if still issue persists. 

 

0 Likes
Highlighted
Honored Contributor.
Honored Contributor.

Hello,

Indeed this is the first attribute in the list as it is the mandatory one, you can see it in the log i put in my post.

In the staging table it's named u_correlation_id so i mapped to this one in the XML file.

Best regards,
Yann Pingot

0 Likes
Highlighted
Super Contributor.. Super Contributor..
Super Contributor..

Just curious if you made any modifications to the servicenowconfig.xml?  I had that same issue before but I forgot exactly what step i did to fix it. 

Example below:

<ci ciType="unix"
stagingTable="ucmdb_integ_ci_linux_server"
targetTable="cmdb_ci_linux_server">
<relations>
<relation childCiType="interface"
type="composition"
childCiTargetTableField="cmdb_ci" />
<relation childCiType="file_system"
type="composition"
childCiTargetTableField="computer" />
<relation childCiType="disk_device"
type="composition"
childCiTargetTableField="computer" />
<relation childCiType="installed_software"
type="composition"
childCiTargetTableField="u_installed_on" />
</relations>

Highlighted
Honored Contributor.
Honored Contributor.

No i didn't touch this file.

Should it be modified ? What is its purpose ?

Since we're using new staging tables & transform maps not OOTB ones do you think it could be the issue ? The only thing i created is a new XML file based on the "push computer 1.0" and a new query, mapped to what we have in Snow.

0 Likes
Highlighted
Super Contributor.. Super Contributor..
Super Contributor..

I think you will need to match your staging and target tables in this file.   An example of what i had to do to send just laptops over to the cmdb_ci_computer table. Can't remember what it was ootb.

<ci ciType="nt"
stagingTable="ucmdb_integ_ci_computer"
targetTable="cmdb_ci_computer">
<relations>
<relation childCiType="interface"
type="composition"
childCiTargetTableField="cmdb_ci" />
<relation childCiType="file_system"
type="composition"
childCiTargetTableField="computer" />
<relation childCiType="disk_device"
type="composition"
childCiTargetTableField="computer" />
<relation childCiType="installed_software"
type="composition"
childCiTargetTableField="u_installed_on" />
</relations>

 

Another thing i did was instead of using correlation_id I created a u_ucmdb_id and mapped the correlation_id to that in ServiceNow since anytime i add a new relationship it will always have "u_" in front so that fixed that problem and i was able to coalesce on that field. Shown Below:

<target_entities>
<source_instance query-name="SNOW Computer Push 1.0" root-element-name="Root">
<target_entity name="ucmdb_integ_ci_computer">
<target_mapping datatype="STRING" name="u_ucmdb_id" value="Root['global_id']"/>
<target_mapping datatype="STRING" name="asset_tag" value="Root['bios_asset_tag']"/>
<target_mapping datatype="BOOLEAN" name="virtual" value="Root['host_isvirtual']"/>
<target_mapping datatype="STRING" name="name" value="Root['display_label']"/>
<target_mapping datatype="STRING" name="u_discover_os_name" value="Root['discovered_os_name']"/>
<target_mapping datatype="STRING" name="u_os_version" value="Root['discovered_os_version']"/>

 

One other thing to check in adapter.properties make sure your class model prefixes are correct.

 

# comma separated list of additional prefixes for ServiceNow tables included in external class model
class.model.table.prefixes=cmdb,cmdb_rel_ci,sys_import_set_row,ucmdb_integ_,cmdb_rel_type,u_ucmdb_integ
# field that UCMDB global_id is mapped to.
class.model.coalesce.field=u_ucmdb_id
# do the incremental class model update for performance reasons or disable it in case you want
# to always get the fresh schema from ServiceNow
class.model.incremental.update=true

 

 

 

0 Likes
Highlighted
Super Contributor.. Super Contributor..
Super Contributor..

Actually the more i think about it maybe your coalesce field in the adapter.properties file. OOTB it is correlation_id

 

May need to change it to u_correlation_id

 

This is how i configured mine.

class.model.coalesce.field=u_ucmdb_id

Highlighted
Respected Contributor.
Respected Contributor.

Yeah tslane is correct. you need to use correlation_id, if you dont want to modify adapter configuration.

Check with your snow expert, if they can add correlation_id in staging table wsdl form. 

or else modify-

class.model.coalesce.field=u_correlation_id

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.