Commodore
Commodore
912 views

Can't get bearer token for REST api anymore (server_error, Unexpected error., exception)

Jump to solution

Hey everyone. Really strange and unpleasant situation here 😞

I was working on some role modifications and I was doing it via REST api. I was tuning up my batch script, as I did several times before. Specificaly I was removing some resources from roles, but I think that's not important. The api access worked great, I used the token successfuly like 100 times.

Now I had everything prepared, but something strange happened, I can not get Bearer token anymore! And I have no idea why 😕 I was not touching anything on the server side, it just started throwing error out of the blue:

This is how I always managed to get it:

curl -X POST -k -H 'Authorization: Basic THISISBASE64rbpmrest:somethingcorrect' 'https://idm.something.something/osp/a/idm/auth/oauth2/token' --data 'grant_type=password&username=cn%3Duaadmin%2Cou%3Dsa%2Co%3Ddata&password=somethingcorrect'

{"error":"server_error","error_description":"Unexpected error.","sub_error":"exception"}

 

Well, not very helpful error. I also tested ` /osp/a/idm/auth/oauth2/grant` - same error.

 

I have tried restarting tomcat, drivers, and even whole box with all idm components. But still them same. catalina.out is silent when I try to get the token, no log message at all.

 

Here are the headers returned:

HTTP/1.1 500 500
Cache-Control: no-cache
Cache-Control: no-store
Connection: close
Content-Type: application/json;charset=UTF-8
Date: Fri, 06 Nov 2020 19:03:56 GMT
Pragma: no-cache
Server: Apache
Strict-Transport-Security: max-age=31536000;includeSubDomains
Transfer-Encoding: chunked
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
x-oidp-forward-response-status: SUCCESS_OK

{
"error": "server_error",
"error_description": "Unexpected error.",
"sub_error": "exception"
}

 

Have anyone else maybe experienced this? Is there some protection against too many requests that I do not know about? And btw, where are the generated tokens saved? I briefly tried to search IDVault and upserapp database, but did not find anythings. 

Thanks for every bit of help.

 

Labels (1)
0 Likes
1 Solution

Accepted Solutions
Vice Admiral
Vice Admiral

This:

Error writing user's OAuth token revocation entries to trust store

might point to problems with writing oidpInstanceData attribute. Can you please check last two entries here:

https://www.netiq.com/documentation/identity-manager-47/identity_apps_admin/data/authentication-issues-troubleshooting.html

 

Kind regards,

Sebastijan

 

View solution in original post

5 Replies
Commodore
Commodore

I have found the error. No idea about the cause & solution (yet)

 

 

idm /opt/netiq/idm/apps/tomcat/logs  # tail -f -n0 osp-idm.2020-11-06.log
Preamble: [OIDP]
Priority Level: WARNING
Java: internal.osp.oidp.service.oauth2.handler.OAuth2Handler.writeTokenRevocationEntries() [772] thread=ajp-nio-127.0.0.1-8109-exec-11
Time: 2020-11-06T20:35:32.562+0100
Log Data: Error writing user's OAuth token revocation entries to trust store.
Class: CoreExceptionWithOutcome
   Logged: false
   Class: LoggableMessage
      Level: SEVERE
      Code: internal.osp.oidp.service.principal.store.SingleAttrStore.putInstanceData() [224]
      Thread: ajp-nio-127.0.0.1-8109-exec-11
      Correlation Id: 88f2cc7f-66fe-4ed8-ac1c-3d3102c320d3
      Text: Error writing instance data for 'cn=uaadmin,ou=sa,o=data'
      Root cause:
null
   Reason: XDAS_OUT_SERVICE_FAILURE

internal.osp.oidp.service.principal.store.SingleAttrStore: SingleAttrStore.java: putInstanceData: 224QA
internal.osp.oidp.service.principal.store.SingleAttrStore: SingleAttrStore.java: writeData: 310
internal.osp.oidp.service.oauth2.handler.OAuth2Handler: OAuth2Handler.java: writeTokenRevocationEntries: 766
internal.osp.oidp.service.oauth2.handler.ResourceOwnerHandlerBase: ResourceOwnerHandlerBase.java: issueROToken: 90
internal.osp.oidp.service.oauth2.handler.ResourceOwnerCredentials: ResourceOwnerCredentials.java: handle: 161
internal.osp.oidp.service.oauth2.handler.Token: Token.java: handle: 52
internal.osp.oidp.service.oauth2.handler.OAuth2Handler: OAuth2Handler.java: processRequest: 454
internal.osp.oidp.service.servlets.handler.AuthenticationServiceRequestHandler: AuthenticationServiceRequestHandler.java: handleRequest: 371
internal.osp.framework.handler.TenantRequestHandler: TenantRequestHandler.java: handleRequest: 156
internal.osp.framework.handler.OSPHandler: OSPHandler.java: handleRequest: 172
internal.osp.framework.servlet.OSPServlet: OSPServlet.java: process: 245
internal.osp.framework.servlet.OSPServlet: OSPServlet.java: doPost: 144
javax.servlet.http.HttpServlet: HttpServlet.java: service: 660
javax.servlet.http.HttpServlet: HttpServlet.java: service: 741
org.apache.catalina.core.ApplicationFilterChain: ApplicationFilterChain.java: internalDoFilter: 231
org.apache.catalina.core.ApplicationFilterChain: ApplicationFilterChain.java: doFilter: 166
org.apache.tomcat.websocket.server.WsFilter: WsFilter.java: doFilter: 53
org.apache.catalina.core.ApplicationFilterChain: ApplicationFilterChain.java: internalDoFilter: 193
org.apache.catalina.core.ApplicationFilterChain: ApplicationFilterChain.java: doFilter: 166
org.apache.catalina.filters.HttpHeaderSecurityFilter: HttpHeaderSecurityFilter.java: doFilter: 126
org.apache.catalina.core.ApplicationFilterChain: ApplicationFilterChain.java: internalDoFilter: 193
org.apache.catalina.core.ApplicationFilterChain: ApplicationFilterChain.java: doFilter: 166
org.apache.catalina.core.StandardWrapperValve: StandardWrapperValve.java: invoke: 202
org.apache.catalina.core.StandardContextValve: StandardContextValve.java: invoke: 96
org.apache.catalina.authenticator.AuthenticatorBase: AuthenticatorBase.java: invoke: 666
org.apache.catalina.core.StandardHostValve: StandardHostValve.java: invoke: 139
org.apache.catalina.valves.ErrorReportValve: ErrorReportValve.java: invoke: 92
org.apache.catalina.valves.AbstractAccessLogValve: AbstractAccessLogValve.java: invoke: 688
org.apache.catalina.core.StandardEngineValve: StandardEngineValve.java: invoke: 74
org.apache.catalina.connector.CoyoteAdapter: CoyoteAdapter.java: service: 343
org.apache.coyote.ajp.AjpProcessor: AjpProcessor.java: service: 432
org.apache.coyote.AbstractProcessorLight: AbstractProcessorLight.java: process: 65
org.apache.coyote.AbstractProtocol$ConnectionHandler: AbstractProtocol.java: process: 868
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor: NioEndpoint.java: doRun: 1,594
org.apache.tomcat.util.net.SocketProcessorBase: SocketProcessorBase.java: run: 49
java.util.concurrent.ThreadPoolExecutor: ThreadPoolExecutor.java: runWorker: 1,149
java.util.concurrent.ThreadPoolExecutor$Worker: ThreadPoolExecutor.java: run: 624
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable: TaskThread.java: run: 61
java.lang.Thread: Thread.java: run: 748

Preamble: [OIDP]
Priority Level: SEVERE
Java: internal.osp.oidp.service.oauth2.handler.RequestHandler.setJsonError() [564] thread=ajp-nio-127.0.0.1-8109-exec-11
Time: 2020-11-06T20:35:32.562+0100
Log Data: Error processing OAuth 2.0 request.: internal.atlaslite.jcce.exception.CoreExceptionWithOutcome: Error writing instance data for 'cn=uaadmin,ou=sa,o=data'
      internal.osp.oidp.service.principal.store.SingleAttrStore: SingleAttrStore.java: putInstanceData: 224

 

I have also checked osp schema, seems fine:

Starting schema update for: osp.sch...
Schema attribute oidpInstanceData already exists and is identical.
The specified optional attributes for the schema class ndsLoginProperties already exist and are identical.

 

Interestingly, the Userapp works. I can login fine.

 

0 Likes
Vice Admiral
Vice Admiral

This:

Error writing user's OAuth token revocation entries to trust store

might point to problems with writing oidpInstanceData attribute. Can you please check last two entries here:

https://www.netiq.com/documentation/identity-manager-47/identity_apps_admin/data/authentication-issues-troubleshooting.html

 

Kind regards,

Sebastijan

 

View solution in original post

Commodore
Commodore

Yeah, thanks a lot for getting me on the right track! Section 43.4.2 and 43.4.3 helped me a lot to understand!

 

In my case, the issue was, that atribute oidpInstanceData was full. My script is kind of dumb, because it was written for just one time use, so I did not bother with reusing tokens and resfreshing them using refresh token, and instead it issues new access token on every run. And while my testing, I did run it many times :))

I deleted the oidpInstanceData from the user and issued new token without issues.

 

Thnkas again. 🙂

Micro Focus Expert
Micro Focus Expert
Commodore
Commodore

Thanks, good stuff 👍

 

I also found this https://github.com/please-openit/oidc-bash-client/blob/master/oidc-client.sh to be good snippet source for various oauth2 stuff in bash 🙂

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.