Cybersecurity
DevOps Cloud
IT Operations Cloud
Overview
Troubleshooting Methodology
Configuration Steps
Example: GroupWise Plugin
Example: Sentinel Plugin
Conclusion
Installation Instructions
Attachments
Supportconfig included in supportutils package has plugin support. This feature allows you to write your own script or program, have supportconfig run it and include it's output as a file with the supportconfig tar ball. This article will include a methodology for writing plugins, installation instructions and two example plugin scripts.
The included supportconfig-plugins package contains all the plugins mentioned in this article and any additional plugins that may be added in the future. Download Now
If you are writing a plugin to help troubleshoot an application or service, I have found it helpful to include some basic information. The information you include should help you determine if the application is installed, configured and running properly. I check the system to answer the following basic questions:
In a nutshell, just copy your script to the /opt/supportconfig/plugins directory and run supportconfig. The detailed steps:
Novell GroupWise is collaboration and productivity software. The GroupWise Supportconfig Plugin gathers important troubleshooting information about GroupWise Agents, Monitor, GWIA and WebAccess.
#!/bin/bash
#############################################################
# Name: GroupWise Supportconfig Plugin
# Description: Gathers important troubleshooting information
# about GroupWise Agents, Monitor, GWIA and
# WebAccess.
#############################################################
SVER=1.00
LOG_LINES=500 #0 means include the entire file
LOG_DIR=/var/log/novell/groupwise
CONF_DIR=/etc/opt/novell/groupwise
CONF_SHARE=/opt/novell/groupwise/agents/share
section_header() {
echo
echo " --------------------------------------------------------- "
printf "#-| %-55s |-#\n" "$*"
echo " --------------------------------------------------------- "
}
cat_log() {
echo "#-> $1"
sed -e '/^[[:space:]]*#/d' -e '/^[[:space:]]*;/d' -e '/^[[:space:]]*\/\//d' -e '/^$/d' $1
echo
}
tail_log() {
echo "#-> $1"
if [ $LOG_LINES -eq 0 ]; then
cat $1
else
tail -${LOG_LINES} $1
fi
echo
}
validate_rpm() {
echo "#-> Validating $THISRPM"
if rpm -V $THISRPM; then
echo " Status: Passed"
else
echo " Status: WARNING"
fi
echo
}
section_header GroupWise Supportconfig Plugin, v${SVER}
section_header RPM List
if RPMLIST=$(rpm -qa | egrep -i "groupwise" | sort); then
echo "$RPMLIST"
else
echo "ERROR: GroupWise RPM Packages not installed"
echo
exit 1
fi
section_header Checking GroupWise Daemons
for i in grpwise grpwise-ma
do
if [ -f /etc/init.d/$i ]; then
chkconfig -l $i
/etc/init.d/$i status
echo
fi
done
section_header Running Processes
ps axwwo user,pid,ppid,%cpu,%mem,vsz,rss,stat,time,cmd | head -1
ps axwwo user,pid,ppid,%cpu,%mem,vsz,rss,stat,time,cmd | grep -E "gwmta|gwpoa|gwinter|gwdva|gwia|gwmon" | grep -v grep
section_header Network Port Status
netstat -nlp | grep -E "gwmta|gwpoa|gwinter|gwdva|gwia|gwmon"
section_header Configuration Files
GWHA=${CONF_DIR}/gwha.conf
cat_log $GWHA
if [ -f $GWHA ]; then
for i in $(\ls -A1 $CONF_SHARE)
do
SFILE=$CONF_SHARE/$i
test -f $SFILE -a -s $SFILE && cat_log $CONF_SHARE/$i
done
else
echo "ERROR: File not found"
echo "File: $GWHA"
echo
fi
section_header Validating RPM List
for THISRPM in $RPMLIST
do
validate_rpm $THISRPM
done
section_header Output Files: Last $LOG_LINES Lines
for OUTFILE in $(find $LOG_DIR -type f | grep output)
do
tail_log $OUTFILE
done
section_header Log Files: Last $LOG_LINES Lines
for GDIR in $(\ls -A1 $LOG_DIR | sort )
do
test "$GDIR" == ".mta" && continue
GLOG="$LOG_DIR/$GDIR"
if [ -d $GLOG ]; then
tail_log $(find $GLOG -type f | sort | grep -E '[[:digit:]]$' | tail -1)
fi
done
Sentinel from Novell is a comprehensive security information and event management (SIEM) solution. The Sentinel Supportconfig Plugin gathers troubleshooting information directly related to Sentinel.
#!/bin/bash
###############################################################
# Name: Sentinel Supportconfig Plugin
# Description: Gathers troubleshooting information directly
# related to Sentinel.
###############################################################
SVER=1.00
LOG_LINES=500
section_header() {
echo
echo " --------------------------------------------------------- "
printf "#-| %-55s |-#\n" "$*"
echo " --------------------------------------------------------- "
}
cat_log() {
echo "#-> $1"
cat $1
echo
}
tail_log() {
echo "#-> $1"
tail -${LOG_LINES} $1
echo
}
section_header Sentinel Supportconfig Plugin, v${SVER}
section_header Sentinel Environment Variables for root
if ! env | grep ^ESEC_; then
echo "ERROR: Sentinel environment variables not set"
echo
fi
if [ -f /etc/init.d/sentinel ]; then
section_header System Daemon Status
chkconfig -l sentinel
/etc/init.d/sentinel status
section_header Network Port Status
echo "# netstat -nap | egrep \"10012|10013|10014\""
netstat -nap | egrep "10012|10013|10014"
fi
SHOME=$ESEC_HOME
if [ -n "$SHOME" ]; then
section_header Sentinel Broker Connections
echo "# $SHOME/bin/list_broker_connections.sh 127.0.0.1 10012"
$SHOME/bin/list_broker_connections.sh 127.0.0.1 10012
section_header JAR File Version Compatibility
for SJAR in $(find $SHOME/lib -type f | grep jar$ | sort)
do
printf "%-60s - %s\n" "$SJAR" "$($SHOME/bin/versionreader.sh $SJAR)"
done
section_header Configuration Files
for CLOG in $(find $SHOME/config -type f | sort)
do
cat_log $CLOG
done
section_header Log Files: Last $LOG_LINES Lines
for SLOG in $(find $SHOME/log -type f | grep 0.log$ | sort)
do
tail_log $SLOG
done
section_header 3rd Party Log Files: Last $LOG_LINES Lines
for TLOG in $(find $SHOME/3rdparty -type f | grep log$ | grep esec | sort)
do
tail_log $TLOG
done
section_header Installation Logs: Last $LOG_LINES Lines
for ILOG in $(find /opt/novell/sentinel6/install_log/ -maxdepth 1 -type f | sort)
do
tail_log $ILOG
done
section_header Cached Event Data Files
echo "# find $SHOME/data/events -type f | xargs ls -l"
find $SHOME/data/events -type f | xargs ls -l
fi
The supportconfig included in the supportutils package is extensible with plugin support. Plugins are any executible program or script, which in turn saves all stdout and stderr output to a log file that is included with the other supportconfig files. The resulting tar ball can be used internally or sent to Novell with your service request information. Plugin support allows you to include any kind of information you deem important to gather. Both the GroupWise and Sentinel supportconfig plugins have been included as an example, and for functional troubleshooting purposes.