Exploding Log Files on NAM 4.2

0 Likes

Why you might be reading this



Detailed log files are great for troubleshooting. But if they get too large, you're going to have a bigger problem than what you're using the log files for - troubleshooting. If the log files consume the disk, you're in a world of hurt, along with unnecessarily alarming the infrastructure team and putting yourself in the "I don't know what my software is doing" spotlight...

Background



When setting up a NAM environment, it's typical to increase the log files on both the Identity servers and Access Gateways either permanently or temporarily. Analyzing these logs is really the only way to solve things that go haywire.

On the Identity servers, NAM uses logrotate to roll up and compress the catalina.out file.

/var/opt/novell/nam/logs/idp/tomcat/catalina.out


On the Access Gateways, logrotate is used to rollup and compress a number of files that may or may not be configured to be written to. The most notable one is the error_log.

/var/log/novell-apache2/error_log


Without logrotate functioning as it's supposed to, both of these files will explode over time. And, out of the box, logrotate is not functioning as it's supposed to. I should emphasize that I've only experienced this on RHEL 7.2. This issue may or may not manifest itself on older versions or SUSE.

Cause



Somewhere along the line, logrotate got smart and prevented logrotate configuration files from running unless they were owned by root. You can see how this might cause a problem. If logrotate is running as a daemon under the root context, anything can happen. A user, either on purpose or by accident, could "logrotate" a file that they do not own. This could mean deleting a file.

Upon installing NAM, the owner of all logrotate configuration files is novlwww. This is the problem.

The Solution



The simple solution: "chown" the logrotate configuration files to be owned by root.

On the Identity Server:
chown root:root /opt/novell/nam/idp/conf/logrotate.conf

On the Access Gateway:
chown root:root /opt/novell/nam/mag/conf/logrotate.conf

On the Admin Consoles:
chown root:root /opt/novell/nam/adminconsole/conf/logrotate.conf

chown root:root /opt/volera/roma/conf/logrotate.conf

chown root:root /etc/opt/novell/tomcat7/logrotate.d/tomcat7

Additionally, you will also need to "chmod" the tomcat7 file as well:
chmod 644 /etc/opt/novell/tomcat7/logrotate.d/tomcat7


Labels:

How To-Best Practice
Comment List
Related
Recommended