Azure AD collector error

Hi everyone.
I have a problem with the Azure AD account and permission collector. When I do a collection test I get the following error message:

[Daas connector returned error during collection: Command failure: Type: find + chunked: [Command failure: Type: find + chunked: [Error collecting using search class: User]

The same thing happens to me with the permission collector for Role and Group class.

What is happening? The connection test is ok.

  • Alas you need to kick up the logging for the com.netiq.daas to figure out what is going on.  (Look in the tomcat\conf for a logging file with com.netiq.daas and com.microfocus.daas classes and kick the tracing up to DEBUG.

  • Hi Geoffrey!
    I have changed the logging level to debug but I see the same error. I do not find new information that gives more details about what is happening.

    can you help me? 

  • If you are on a more current version (3.6.2 maybe?)  there is a logging menu item under configuration in the UI.  When you add and make changes there, it overrides the file based logging files.  It takes a few moments for the services to figure out the new logging levels, but its 1000 times better than restarting tomcat.   Also note that each service module or context has its own set of logging, so in this case you'd switch to daas, then ensure the com.netiq.daas or com.microfocus.daas is cranked up to debug or trace if you dare.

    --Jim

  • I did exactly that. 

    [FINE] 2021-09-29 13:39:11.948 [com.netiq.daas.daaservice.ServiceMap] [DAAS] Received request to load service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3fa01506f0dbece7ce
    [FINE] 2021-09-29 13:39:11.948 [com.netiq.daas.daaservice.ServiceMap] [DAAS] Loaded service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3f201506f8dbece7ce, load count: 1
    [FINE] 2021-09-29 13:39:12.004 [com.netiq.daas.daaservice.ServiceProvideriap] [DAAS] Collection cleaner running...
    [FINE] 2021-09-29 13:39:12.004 [com.netig.daas.common.SrvInstance] [DAAS] New service instance. TTL: 60
    [FINE] 2021-09-29 13:39:12.004 [com.netig.daas.common.SrvInstance] [DAAS] Reset timeout for service instance to TTL: 60
    [FINE] 2021-09-29 13:39:12.707 [com.netiq.daas.daaservice.ServiceMap] [DAAS] Received request to unload service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3f201506f8dbece7ce
    [FINE] 2021-09-29 13:39:25.707 [com.netiq.daas.daaservice.ServiceMap] [DAAS] Decremented load count on service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3f201506f8dbece7ce, load count: null
    [FINE] 2021-09-29 13:39:25.170 [com.netiq.daas.Haaservice.ServiceMap] [DAAS] Received request to load service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3f201506f8dbece7ce
    [FINE] 2021-09-29 13:39:25.171 [com.netiq.daas.daaservice.ServiceMap] [DAAS] Loaded service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3f201506f8dbece7ce, load count: 1
    [FINE] 2021-09-29 13:39:25.229 [com.netiq.daas.daaservice.ServiceProvideriap] [DAAS] Collection cleaner running...
    [FINE] 2021-09-29 13:39:25.229 [com.netig.daas.common.SrvInstance] [DAAS] New service instance. TTL: 60
    [FINE] 2021-09-29 13:39:25.229 [com.netiq.daas.common.SrvInstance] [DAAS] Reset timeout for service instance to TTL: 60
    [FINE] 2021-09-29 13:39:25.946 [com.netiq.daas.daaservice.ServiceMap] [DAAS] Received request to unload service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3f201506f8dbece7ce
    [FINE] 2021-09-29 13:39:25.946 [com.netiq.daas.daaservice.ServiceMap] [DAAS] Decremented load count on service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3f201506f8dbece7ce, load count: null
    [FINE] 2021-09-29 13:39:52.361 [com.netiq.iac.server.dtp.TestDataProcessingTask] [IG-SERVER] Successfully invoked data test collection execution service for test collection id=485, collector id=17
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.ServiceView] [DAAS] InputTransformsMap: 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.ServiceView] [DAAS]      {disabled=Azure_Disabled Convert} 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed 
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] view-name or app-name is null. Mapping not allowed
    [FINE] 2021-09-29 13:39:53.593  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] SchemaMap: 
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS]    m_appKeyMap:
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS]       {objectId=[accountId, userLink], displayName=[name], userPrincipalName=[description], accountEnabled=[disabled]} 
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS]    m_viewKeyMap:
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS]       {accountId=objectId, name=displayName, description=userPrincipalName, disabled=accountEnabled, userLink=objectId}
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS]    m_staticValueMap:
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS]       {privileged=false}
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS]    m_compositeMap:
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS]       {}
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.SchemaMap] [DAAS] SchemaMap:
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.ServiceView] [DAAS] Query Object: {"search-class": "User"}
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.ServiceView] [DAAS] service: com.netiq.daas.daaservice.util.Servic019Sc62c9, view: account 
    [FINE] 2021-09-29 13:39:53.594  [com.netiq.daas.daaservice.util.ServiceView] [DAAS]      view collectionQuery: {
     "type": "find",
     "command": { 
        "search-class": "User",
        "read-attrs": [
          "accountld",
          "name", 
          "description",
          "disabled",
          "userLink",
          "type", 
          "userld",
          "state", 
          "connAcctld",
          "cost", 
          "risk", 
          "lastLogin",
          "aliases",
          "collectedUsersOwners",
          "idmAccountld"
        ]
      }
    }
    [FINE] 2021-09-29 13:39:53.594[com.netiq.daas.daaservice.util.Service] [DAAS] ADDING view: account to service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3fa01506f0dbece7ce
    [FINE] 2021-09-29 13:39:53.595[com.netiq.daas.daaservice.ServiceMap] [DAAS] Received request to load service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3fa01506f0dbece7ce
    [FINE] 2021-09-29 13:39:53.595[com.netiq.daas.daaservice.ServiceMap] [DAAS] Loaded service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3fa01506f0dbece7ce, load count: 1
    [FINE] 2021-09-29 13:39:53.723[com.netiq.daas.daaservice.util.QueryProcessor] [DAAS] )JSON Rawrequest: {"SIZE":10000}
    [FINE] 2021-09-29 13:39:53.723[com.netiq.daas.daaservice.util.QueryProcessor] [DAAS] Got NEWQUERY command: {
      "type": "find",
      "command": { 
        "search-class": "User",
        "read-attrs": [
          "accountld",
          "name", 
          "description",
          "disabled",
          "userLink",
          "type", 
          "userld",
          "state", 
          "connAcctld",
          "cost", 
          "risk", 
          "lastLogin",
          "aliases",
          "collectedUsersOwners",
          "idmAccountld"
        ]
      }
    }
    [FINE] 2021-09-29 13:39:53.724 [com.netiq.daas.daaservice.util.QueryProcessor] [DAAS] Got generated chunked command: {
      "type": "find",
      "command": { 
        "search-class": "User",
        "read-attrs": [
          "accountld",
          "name", 
          "description",
          "disabled",
          "userLink",
          "type", 
          "userld",
          "state", 
          "connAcctld",
          "cost", 
          "risk", 
          "lastLogin",
          "aliases",
          "collectedUsersOwners",
          "idmAccountld"
        ]
      }.
     "SIZE": 10000
    }
    [FINE] 2021-09-29 13:39:53.724 [com.netiq.daas.daaservice.util.QueryProcessor] [DAAS] JSON Rawrequest. Command: 
    [FINE] 2021-09-29 13:39:53.724 [com.netiq.daas.daaservice.util.QueryProcessor] [DAAS]    {
      "type": "find",
      "command": { 
        "search-class": "User",
        "read-attrs": [
          "accountld",
          "name", 
          "description",
          "disabled",
          "userLink",
          "type", 
          "userld",
          "state", 
          "connAcctld",
          "cost", 
          "risk", 
          "lastLogin",
          "aliases",
          "collectedUsersOwners",
          "idmAccountld"
        ]
      }.
     "SIZE": 10000
    } 
    [FINE] 2021-09-29 13:39:53.724 [com .netiq.daas.daaservice.ServiceProviderMap] [DAAS] Collection cleaner running... 
    [FINE] 2021-09-29 13:39:53.724 [com .netiq.daas.common.SrvInstance] [DAAS] New service instance. TTL: 60
    [FINE] 2021-09-29 13:39:53.724 [com .netiq.daas.daaservice.util.QueryProcessor] [DAAS] calling preProcess...
    [FINE] 2021-09-29 13:39:53.724 [com .netiq.daas.daaservice.util.Transformation] [DAAS] Got REQUEST schema map...
    [FINE] 2021-09-29 13:39:53.724 [com .netiq.daas.daaservice.util.Transformation] [DAAS] requestMap: in viewCommand: { 
     "search-class": "User",
     "read-attrs": [
      "accountld",
      "name", 
      "description",
      "disabled",
      "userLink",
      "type", 
      "userld",
      "state", 
      "connAcctld",
      "cost", 
      "risk", 
      "lastLogin",
      "aliases",
      "collectedUsersOwners",
      "idmAccountId" 
     ]
    }
    [FINE] 2021-09-29 13:39:53.725 [com .netiq.daas.daaservice.util.Transformation] [DAAS] requestMap: OUT viewCommand: { 
     "search-class": "User",
     "read-attrs": [
      "objectld",
      "displayName",
      "userPrincipalName",
      "accountEnabled"
     ]
    }
    FINE] 2021-09-29 13:39:53.725 [com.netiq.daas. daaservice.util.Transformation] [DAAS] preprocess complete: {"search-class":"User","read-attrs":["objectId","displayName","userPrincipalName","accountEnabled"]} 
    [FINE] 2021-09-29 13:39:53.725 [com.netiq.daas. daaservice.util.QueryProcessor] [DAAS] calling executeJSONChunkRequest: {
      "search-class": "User",
      "read-attrs": [
        "objectId",
        "displayName",
        "userPrincipalName",
        "accountEnabled"
     ]
    }
    [FINE] 2021-09-29 13:39:53.725 [com.netiq.daas. common.Srvinstance] [DAAS] Reset timeout for service instance to TTL: 60
    [FINE] 2021-09-29 13:39:53.726 [com.netiq.daas. azureservice.AzureService] [DAAS] request schemaList: [objectId, displayName, userPrincipalName, accountEnabled]
    [FINE] 2021-09-29 13:39:54.236 [com.netiq.daas. azureservice.AzureService] [DAAS] Getting chunked results:
    [FINE] 2021-09-29 13:39:55.095 [com.netiq.daas. common.Srvinstance] [DAAS] Reset timeout for service instance to TTL: 60
    [INFO] 2021-09-29 13:39:55.095 [com.netiq.daas. azureservice.AzureService] [DAAS] Received service shutdown from DaaS
    [FINE] 2021-09-29 13:39:56.447 [com.netiq.daas. daaservice.ServiceMap] [DAAS] Received request to unload service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3fa01506f0dbece7ce
    [FINE] 2021-09-29 13:39:56.447 [com.netiq.daas. daaservice.ServiceMap] [DAAS] Decremented load count on service: AzureAccountTemplate-10-17-8c6e97e0d2b74b3fa01506f0dbece7ce,  load count: null
    [SEVERE] 2021-09-29 13:39:56.451 [com.netiq.iac .persistence.dcs.dce.thread.TestDataCollectionServiceThread] [IG-DTP] DaaS connector returned error during collection: Command failure: Type: find+chunked: [Error collecting using search class: User]]
    

  • I don't see a results count in there (I'm not familiar with the Azure collector to know if it does that, but I'd expect to see results from the query).  It looks like your filter is for User class objects --> in Azure do objects have "ADUser" class?   That might be why the results are empty -- or maybe I'm wrong and azure users are of "User" class.

    --Jim

  • Hi gimesierra,

    Did you ever get this working? I tired to set up the Azure AD User account collector for IG 3.7, with the same results as reported by you. Enabling DEBUG level logging does not reveal any additional information. Couldn't find any documentation about the collector either, so I had to guess things like the permissions required and parameter values. Connection test says "ok".

  • I've seen the same error message, and I found that granting additional rights to the service account that I'm using fixed it.  I think the error is generated when there are 0 results.  One way to get there is by not having the appropriate rights to see any Users.   I'm sure there are other ways to generate the same error too.

    --Jim

  • Could be. I'm connecting to a test tenant and had Directory.Read.All and User.Read.All grants (Microsoft Graph) for the app. Since I'm trying to collect just the accounts (for now), I figured this should be more than enough. It would be really helpful if there would be additional debug logging but raising the daas log levels to debug doesn't give any more information.

  • I had added User.Read, User.ReadBasiclAll, and User.Read.All under Microsoft Graph.   And then you have to do the admin consent thing for those.

    Also, because I'm a terrible IT guy, I additionally added a bunch of Windows Azure Active Dreictory permissions, but I woudl like to think the collector only uses graph.   So, if you end up thinking you need more permissions, I can list out what I've done with those as well.

    --Jim

  • I thought about the legacy Azure AD Graph API permissions as well, but since it's already deprecated, I figured out this couldn't be the reason. Now that I think about this again, I'm pretty sure this is exactly the reason. The URL that the collector uses by default is https://graph.windows.net/, and this is actually the URL of the legacy Azure AD Graph API...

    I think this also reveals why this is poorly documented. Why bother, since the used API is going away.

    Granting old AAD Graph API permissions is no longer possible through the admin portal. I think it could still be done through powershell however. Maybe I'll try it, even though this collector clearly needs to be rewritten by NetIQ to use the Graph API.