Service Manager 9.41 and SSO

Hi

 

I am facing SSL issue in version 9.41. I just upgraded to application rte version on test to version 9.41 and it runs fine without SSL, but when i want to use SSL and SSO, it gives me error handshake faliure on the client and in the log it says "RTE E GetPreference DOS attack detected! Session will be terminated".

 

Kindly help

Tags:

  • You need to debug SSL in detail:


     - add parameter in sm.ini or sm.cfg
       JVMOption0:-Djavax.net.debug=ssl
      
     - log without this option
      6964(  2412) 01/14/2015 09:49:41  RTE I SOAP client information scguiwswt 9.33.0035 () at 10.10.10.10
      6964(  6080) 01/14/2015 09:49:41 JRTE E Remote host(10.10.10.10/10.10.10.10) is not a trusted client
      6964(  6080) 01/14/2015 09:49:41 JRTE W Send error response: Client Authentication failed.
     
     - log with this option
      2368(  5288) 01/14/2015 10:24:40  RTE I SOAP client information scguiwswt 9.33.0035 () at 10.10.10.10
      2368(  4944) 01/14/2015 10:24:40 JRTE I SSL: Verifying client's certificate...
      2368(  4944) 01/14/2015 10:24:40 JRTE I SSL: Got certificate from request!
      2368(  4944) 01/14/2015 10:24:40 JRTE I SSL: Verifying client host name...
      2368(  4944) 01/14/2015 10:24:40 JRTE I SSL: Certificate's common name is domainnamedm002.domainname302d.com
      2368(  4944) 01/14/2015 10:24:40 JRTE I SSL: request was sent from 10.10.10.10
      2368(  4944) 01/14/2015 10:24:40 JRTE I SSL: request was sent from IP address 10.10.10.10
      2368(  4944) 01/14/2015 10:24:40 JRTE I SSL: Trying to resolve remote host name from IP 10.10.10.10...
      2368(  4944) 01/14/2015 10:24:40 JRTE I SSL: Got a host name "domainnamedm002" from IP/10.10.10.10
      2368(  4944) 01/14/2015 10:24:40 JRTE E Remote host(10.10.10.10/10.10.10.10) is not a trusted client
      2368(  4944) 01/14/2015 10:24:40 JRTE I Certificate's common name(domainnamedm002.domainname302d.com) is different from the client's host name(10.10.10.10).
      2368(  4944) 01/14/2015 10:24:40 JRTE W Send error response: Client Authentication failed.

     

    Also share the sm.ini and sm.cfg files.

     

    BR,

    Alex

  • Just wanted to write down for a future reference (because somebody else will be hit by this), that after SM 9.41 patch #6 bunch of algorithms (please refer to SM 9.41 patch #6 release notes for a complete list ) are classified as compromised ones. Also DH (<keysize < 768) nd RSA (keysize < 2048) are classified as compromised.

    If your certificate is secured by one of those blocked algorithms, Service Manager bluntly says in sm.log:

    GetPreference DOS attack detected! Session will be terminated.

    Webtier's log just tells that the handshake failed.

    Does not really tell why the session was terminated and it is hard to root out what went wrong.

  • Hi did you solve the problem?

    Can you help with the same problem?

  • Well, you have to options:

    • As the release notes of SM 9.41 patch #6 says, it's possible to edit the list of compromised algorithms (a file in Server\RUN). Remove the blocked algorithm or adjust keysize length. Please refer to SM 9.41 patch #6 for more information, I don't have it right now in my hands
    • Create a new certificate/s with a longer keysize or more modern algorithm

    Of course solution #1 should only be taken as a temporary solution. Outdated certificates should be regenerated as soon as possible. (Of course the test systems are a different case, but it's a good rehearsal to update those as well.)