uCMDB-SNOW Integration via Push Adapter

I receive below error always while pushing CIs to snow. Connection is fine. But i can insert entry in snow via SOAPUI with same wsdl link and credentials. Let me know for any idea?

 

2019-07-29 11:16:46,979 TRACE [AdHoc:AD_HOC_TASK_PATTERN_ID-1129-1564391806685] - Adding u_imp_ucmdb_server RTN to the u_imp_ucmdb_server cache. ResultTreeNode. QueryNodeName: u_imp_ucmdb_server
ExternalCiId:***{UCMDB nt 1 internal_id=STRING=4a33dfc5e50e7497835c27875696ff7d }***
class name: nt
isCmdbId: true
cmdbId: 4a33dfc5e50e7497835c27875696ff7d
Properties:
internal_id=4a33dfc5e50e7497835c27875696ff7d
Properties:
u_name = testci
u_rwe_correlationid = 4a33dfc5e50e7497835c27875696ff7d,

2019-07-29 11:16:46,979 ERROR [AdHoc:AD_HOC_TASK_PATTERN_ID-1129-1564391806685] - Ignoring u_imp_ucmdb_server RTN tree because of invalid key. ExternalCiId:***{UCMDB nt 1 internal_id=STRING=4a33dfc5e50e7497835c27875696ff7d }***
class name: nt
isCmdbId: true
cmdbId: 4a33dfc5e50e7497835c27875696ff7d
Properties:
internal_id=4a33dfc5e50e7497835c27875696ff7d

2019-07-29 11:16:46,979 WARN [AdHoc:AD_HOC_TASK_PATTERN_ID-1129-1564391806685] - Unable to create global CI ID for RTN. Using the original ExternalCiId: ExternalCiId:***{UCMDB nt 1 internal_id=STRING=4a33dfc5e50e7497835c27875696ff7d }***
class name: nt
isCmdbId: true
cmdbId: 4a33dfc5e50e7497835c27875696ff7d
Properties:
internal_id=4a33dfc5e50e7497835c27875696ff7d

java.lang.NullPointerException
at com.hpe.ucmdb.adapters.util.UcmdbUtils.createGlobalExternalCiId(UcmdbUtils.java:70)
at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.reportAdditionStatus(ServiceNowPusher.java:623)
at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.addOrUpdateRtn(ServiceNowPusher.java:229)
at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.pushTreeNodes(ServiceNowPusher.java:203)
at com.hp.ucmdb.adapters.snow.ServiceNowGenericAdapter.pushTreeNodes(ServiceNowGenericAdapter.java:179)
at com.hp.ucmdb.adapters.GenericAdapter.pushToRemoteDataSource(GenericAdapter.java:575)
at com.hp.ucmdb.adapters.GenericAdapter.updateData(GenericAdapter.java:408)
at com.hp.ucmdb.dataAccess.manager.DirectBasicDataAdapterWrapper.updateData(DirectBasicDataA

  • Hi  ,

    In the last year I have extensively worked on the ServiceNow adapter and I can give you some outline of how to debug it:

    1. Make sure you are using REST and not SOAP. It is far faster. In adapter.properties set:

    connector.class=com.hpe.ucmdb.adapters.snow.connector.ServiceNowRestConnector
    # connector.class=com.hpe.ucmdb.adapters.snow.connector.ServiceNowSoapConnector
    push.connector.class=com.hpe.ucmdb.adapters.snow.connector.ServiceNowRestConnector

     

    2. Enable debugging of servicenow adapter. This will show you the REST queries sent to ServiceNow and what has been received as a response. Ammend fcmdb.properties in DataFlowProbe\conf\log with:

     

    #### ServiceNow Custom Configuration

    log4j.category.servicenow.generic.adapter=TRACE,servicenow.generic.adapter.appender
    log4j.appender.servicenow.generic.adapter.appender=com.mercury.topaz.cmdb.shared.base.log.BetterRollingFileAppender
    log4j.appender.servicenow.generic.adapter.appender.File=${logs.dir}/servicenow.generic.adapter.log
    log4j.appender.servicenow.generic.adapter.appender.MaxFileSize=10240KB
    log4j.appender.servicenow.generic.adapter.appender.MaxBackupIndex=${def.files.backup.count}
    log4j.appender.servicenow.generic.adapter.appender.layout=org.apache.log4j.PatternLayout
    log4j.appender.servicenow.generic.adapter.appender.layout.ConversionPattern=${msg.layout}

     

    3. Check the import set and transform map history logs in ServiceNow. If the data is properly sent there, you will be able to see what is the reason for rejection.

     

    Regards,

    Petko Popadiyski

    Freelance Microfocus CMS UCMDB Consulting

  • Hello, Thanks for your reply. I have already done first and second step. I cannot perform third step, as we don't have access to snow. Moreover, data is not reaching to snow staging tables and that's confirmed.

    Let me know for any suggestions please.

     

    Regards,

    Rahul

  • if you have done 1 and 2, you have logs with the requests to ServiceNow. what are the request payloads and what are the responses?

  • Hello,

    I'm facing the same issue when trying to push into a SNOW instance.

    The TQL is fairly simple, and i've reduced the target_mapping fields to the strict minimum (global_id -> correlation_id and name -> name), but with no luck.

    I've also activated traces on the probe and it seems that no call to SNOW is even done, so it's probably an issue on CMDB side, here is the output :

    2020-07-17 17:17:25,157  TRACE  [AdHoc:AD_HOC_TASK_PATTERN_ID-688-1594999044252] - Adding u_imp_microfocus_server RTN to the u_imp_microfocus_server cache.     ResultTreeNode. QueryNodeName: u_imp_microfocus_server
            ExternalCiId:***{UCMDB
    unix
    1
    internal_id=STRING=450339ff03f73f448ee46f66ad1c9fd6
    }***
                class name: unix
                isCmdbId: true
                cmdbId: 450339ff03f73f448ee46f66ad1c9fd6
                Properties:
                    internal_id=450339ff03f73f448ee46f66ad1c9fd6
            Properties:
                u_correlation_id = 406bb305cb7f8d89a772984d3058cc9d
                u_name = cms2lnx0002, 
    
    2020-07-17 17:17:25,172  ERROR  [AdHoc:AD_HOC_TASK_PATTERN_ID-688-1594999044252] - Ignoring u_imp_microfocus_server RTN tree because of invalid key.    ExternalCiId:***{UCMDB
    unix
    1
    internal_id=STRING=450339ff03f73f448ee46f66ad1c9fd6
    }***
            class name: unix
            isCmdbId: true
            cmdbId: 450339ff03f73f448ee46f66ad1c9fd6
            Properties:
                internal_id=450339ff03f73f448ee46f66ad1c9fd6
    
    2020-07-17 17:17:25,172  WARN   [AdHoc:AD_HOC_TASK_PATTERN_ID-688-1594999044252] - Unable to create global CI ID for RTN. Using the original ExternalCiId: ExternalCiId:***{UCMDB
    unix
    1
    internal_id=STRING=450339ff03f73f448ee46f66ad1c9fd6
    }***
        class name: unix
        isCmdbId: true
        cmdbId: 450339ff03f73f448ee46f66ad1c9fd6
        Properties:
            internal_id=450339ff03f73f448ee46f66ad1c9fd6
    
    java.lang.NullPointerException
    	at com.hpe.ucmdb.adapters.util.UcmdbUtils.createGlobalExternalCiId(UcmdbUtils.java:70)
    	at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.reportAdditionStatus(ServiceNowPusher.java:535)
    	at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.addOrUpdateRtn(ServiceNowPusher.java:223)
    	at com.hpe.ucmdb.adapters.snow.push.ServiceNowPusher.pushTreeNodes(ServiceNowPusher.java:199)
    	at com.hp.ucmdb.adapters.snow.ServiceNowGenericAdapter.pushTreeNodes(ServiceNowGenericAdapter.java:178)
    	at com.hp.ucmdb.adapters.GenericAdapter.pushToRemoteDataSource(GenericAdapter.java:817)
    	at com.hp.ucmdb.adapters.GenericAdapter.processMapping(GenericAdapter.java:554)
    	at com.hp.ucmdb.adapters.GenericAdapter.updateData(GenericAdapter.java:526)
    	at com.hp.ucmdb.dataAccess.manager.DirectBasicDataAdapterWrapper.updateData(DirectBasicDataAdapterWrapper.java:434)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at com.hp.ucmdb.discovery.probe.processor.GenericPushProbeRequestProcessor.processRequest(GenericPushProbeRequestProcessor.java:36)
    	at com.hp.ucmdb.discovery.probe.processor.GenericPushProbeRequestProcessor.processRequest(GenericPushProbeRequestProcessor.java:13)
    	at com.hp.ucmdb.discovery.probe.processor.AbstractProbeProcessor.process(AbstractProbeProcessor.java:56)
    	at com.hp.ucmdb.discovery.probe.processor.AbstractProbeProcessor.process(AbstractProbeProcessor.java:19)
    	at com.hp.ucmdb.discovery.probe.agents.probemgr.adhoctasks.AdHocProbeRequestOperation.performAction(AdHocProbeRequestOperation.java:64)
    	at com.hp.ucmdb.discovery.probe.agents.probemgr.taskdispatcher.AdHocTaskDispatcher.dispatchTask(AdHocTaskDispatcher.java:70)
    	at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
    	at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
    	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
    	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
    	at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
    	at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
    	at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
    	at javax.management.StandardMBean.invoke(StandardMBean.java:405)
    	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
    	at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
    	at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:298)
    	at org.springframework.jmx.access.MBeanClientInterceptor.doInvoke(MBeanClientInterceptor.java:417)
    	at org.springframework.jmx.access.MBeanClientInterceptor.invoke(MBeanClientInterceptor.java:366)
    	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    	at com.sun.proxy.$Proxy66.dispatchTask(Unknown Source)
    	at com.hp.ucmdb.discovery.probe.agents.probegw.managementtasks.adhoctasks.AdhocThread.run(AdhocThread.java:54)
    	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
    	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
    	at java.lang.Thread.run(Thread.java:748)
    
    

     

    Of course i've triple checked the inbound service name and properties on SNOW side, and there is nothing in import sets logs since no call is done.

    Any clue or thing i can do to debug this ?

    Thanks,
    Best regards,
    Yann Pingot

  • Please check first, if your xml file have correlation_id attribute for snow.  Are you using staging table to push data?

    Let me know, if still issue persists. 

     

  • Hello,

    Indeed this is the first attribute in the list as it is the mandatory one, you can see it in the log i put in my post.

    In the staging table it's named u_correlation_id so i mapped to this one in the XML file.

    Best regards,
    Yann Pingot

  • Just curious if you made any modifications to the servicenowconfig.xml?  I had that same issue before but I forgot exactly what step i did to fix it. 

    Example below:

    <ci ciType="unix"
    stagingTable="ucmdb_integ_ci_linux_server"
    targetTable="cmdb_ci_linux_server">
    <relations>
    <relation childCiType="interface"
    type="composition"
    childCiTargetTableField="cmdb_ci" />
    <relation childCiType="file_system"
    type="composition"
    childCiTargetTableField="computer" />
    <relation childCiType="disk_device"
    type="composition"
    childCiTargetTableField="computer" />
    <relation childCiType="installed_software"
    type="composition"
    childCiTargetTableField="u_installed_on" />
    </relations>

  • No i didn't touch this file.

    Should it be modified ? What is its purpose ?

    Since we're using new staging tables & transform maps not OOTB ones do you think it could be the issue ? The only thing i created is a new XML file based on the "push computer 1.0" and a new query, mapped to what we have in Snow.

  • I think you will need to match your staging and target tables in this file.   An example of what i had to do to send just laptops over to the cmdb_ci_computer table. Can't remember what it was ootb.

    <ci ciType="nt"
    stagingTable="ucmdb_integ_ci_computer"
    targetTable="cmdb_ci_computer">
    <relations>
    <relation childCiType="interface"
    type="composition"
    childCiTargetTableField="cmdb_ci" />
    <relation childCiType="file_system"
    type="composition"
    childCiTargetTableField="computer" />
    <relation childCiType="disk_device"
    type="composition"
    childCiTargetTableField="computer" />
    <relation childCiType="installed_software"
    type="composition"
    childCiTargetTableField="u_installed_on" />
    </relations>

     

    Another thing i did was instead of using correlation_id I created a u_ucmdb_id and mapped the correlation_id to that in ServiceNow since anytime i add a new relationship it will always have "u_" in front so that fixed that problem and i was able to coalesce on that field. Shown Below:

    <target_entities>
    <source_instance query-name="SNOW Computer Push 1.0" root-element-name="Root">
    <target_entity name="ucmdb_integ_ci_computer">
    <target_mapping datatype="STRING" name="u_ucmdb_id" value="Root['global_id']"/>
    <target_mapping datatype="STRING" name="asset_tag" value="Root['bios_asset_tag']"/>
    <target_mapping datatype="BOOLEAN" name="virtual" value="Root['host_isvirtual']"/>
    <target_mapping datatype="STRING" name="name" value="Root['display_label']"/>
    <target_mapping datatype="STRING" name="u_discover_os_name" value="Root['discovered_os_name']"/>
    <target_mapping datatype="STRING" name="u_os_version" value="Root['discovered_os_version']"/>

     

    One other thing to check in adapter.properties make sure your class model prefixes are correct.

     

    # comma separated list of additional prefixes for ServiceNow tables included in external class model
    class.model.table.prefixes=cmdb,cmdb_rel_ci,sys_import_set_row,ucmdb_integ_,cmdb_rel_type,u_ucmdb_integ
    # field that UCMDB global_id is mapped to.
    class.model.coalesce.field=u_ucmdb_id
    # do the incremental class model update for performance reasons or disable it in case you want
    # to always get the fresh schema from ServiceNow
    class.model.incremental.update=true

     

     

     

  • Actually the more i think about it maybe your coalesce field in the adapter.properties file. OOTB it is correlation_id

     

    May need to change it to u_correlation_id

     

    This is how i configured mine.

    class.model.coalesce.field=u_ucmdb_id