Kerberos authentication may fail with Access Manager Identity Server for users with large group memberships
Access Manager allows you to use Kerberos as a method for authenticate users accessing protected resources (see the Novell Access Manager documentation for details). This is a great feature but can also significantly increase the complexity of your Access Manager deployment.
Background: Access Manager SSO is based on Kerberos authentication. Each user that logs in to the corporate domain is issued a Kerberos ticket by a domain controller. This ticket is requested by the Access Manager server at the time a SSO site is accessed. The browser presents the SPNEGO token in the HTTP header of the request for the SSO enabled resource. This SPNEGO token is a wrapper of the Microsoft Windows Kerberos token. Microsoft Windows includes the user’s PAC information in the Kerberos token. The more security groups that the user belongs to, the more PAC information is inserted in the Kerberos token, and the larger the SPNEGO token becomes. Apache Tomcat 5 limits the size of any acceptable HTTP header to be 8K by default. In Microsoft Windows domains having many groups, and with user membership in many groups, the size of the user’s SPNEGO token may exceed the 8K limit.
The Symptom: When a user requests access to a protected resource they are re-directed to the Identity Server for authentication. The Identity server then requests the browser to re-request the page and this time include the SPNEGO token in the HTTP header of the request. If the browser sends a request that has a header size larger than the default 8k limit, Apache Tomcat abruptly closes the TCP connection. To the user, this will simply appear as "Page cannot be displayed" in Internet Explorer and Firefox. Since Tomcat is brokering the communications between the browser and Access Manager, there will be no errors or any indications whatsoever of a problem in the Access Manager logs. Also, by default, Tomcat does not log why the connection was closed.
Verify the Problem: There are two thing you can do to verify that this is the actual problem you are having. The first is to do a packet trace between the browser and Identity Server. You should see a GET packet that includes the SPNEGO token listed under AUTHORIZATION in the HTTP header. If the header of that GET packet is over 8kb, you will see this issue.
The second thing you can do is get the TOKENSZ utility from Microsoft. Run the following command on the workstation that is having difficulty authenticating to the Identity Server...
In the output, you will be looking for <strong>MaxToken</strong>. This is the size of this particular user's Kerberos Token. Keep in mind that even if the size of this token isn't over 8192 bytes, it's the COMBINATION of this token and the rest of the HTTP header (which will vary in your environment) that will be limited to 8192 bytes.
The Solution: You can adjust the HTTP header size that Apache Tomcat will accept (and thus pass along over to Access Manager for authentication). To do this you will need to edit the following file (valid for Access Manager 3.1.1 and most likely any 3.x version)...
Within this file, look for a line starting with the following...
At the end of this line, add the following parameter...
In our environment we chose a 32k max header size in the hopes that we will never have to deal with this issue again but you can also do something like 16384 for a 16k maximum header size if you choose. Tomcat will allocate 2 buffers of 32kb per connection (one for input and one for output). Yes, your server will use up a bit more memory in this configuration, but considering the connection to the IDP are usually short lived, it shouldn't be much of an issue.
Once you've made this change, save the file and bounce Tomcat or bounce the server to change the running configuration.