With more external apps relying on SOAP to communicate with the POA, how do you monitor the POA to make sure your SOAP threads aren't getting tied up? You can't (presently) get the info from snmp queries, so this script uses the http interface to poll. It is able to retrieve counts of SOAP, IMAP, POP, C/S, MsgHandler, and GWCheck threads, and return with a status of OK, WARNING, or CRITICAL based on configurable threshholds.

To use this script with nagios, download and install into your plugins directory on the server to be monitored, make sure it is executable by the nagios user.

It can also be run stand-alone, or called by another script; see examples below.

Usage: check_gw_threads -H server -r thread_type [OPTIONS]

Options:
  -H host		name or address of host to test
  -r thread_type	one of soap, imap, pop, cs, msg, gwck
  -p port		poa http port (default=7181)
  -U username	poa http username
  -P password	poa http password
  -W wgetrc	complete path to alternate wgetrc file
  -n number	minumum number of idle threads needed for OK status (default=0)
  -i number	minimum percent of idle threads needed for OK status (default=0)
  -d number	max elapsed time in seconds for any active thread (default=60480)
  -s		use https instead of http
  -h, -?		this help message

  Exits WARNING  if -n or -i or -d exceeded.
  Exits CRITICAL if (-n or -i) AND (-d) exceeded.

 Typically you'd specify either -n or -i, and -d.

 To avoid specifying http username and password on the command line, they can be set
 in the ~/.wgetrc file of the user running this script (typically nagios).  You also
 will need to set 'auth_no_challenge = on'.  However, this may affect other scripts
 which use wget.  The preferred way is to specify an alternate wgetrc file location 
 with the -W option.

I define the command like:

define command{
        command_name    check_gw_threads
        command_line    $USER1$/check_gw_threads.sh -H $HOSTADDRESS$ -W $USER1$/check_gw_threads.wgetrc $ARG1$ $ARG2$ $ARG4$ $ARG5$ $ARG6$ $ARG7$ $ARG8$ $ARG9$ $ARG10$ $ARG11$ $ARG12$ $ARG13$ $ARG14$ $ARG15$ $ARG16$
        }

Then configure the service something like:

define service{
        use                     gwt-service
        service_description     POA Soap Threads
        check_command           check_gw_threads!-r soap!-n 2!-d 300!-i 1
        service_groups          grpwisegroup
        host_name               ngwnameserver
}

Change your template, service_groups, host_name as appropriate.

Then I also need to have in the nagios plugins directory the file check_gw_threads.wgetrc which contains:

http_user = httpusername
http_password = httppassword
auth_no_challenge = on

(replace httpusername and httpassword with appropriate values for your site).

Examples:

> sh check_gw_threads.sh -H ngwnameserver -r soap -U httpusername -P httppassword
POA SOAP Threads: 6 Total, 6 Idle, 100% Idle, Oldest 0 seconds

> sh check_gw_threads.sh -H ngwnameserver -r cs -U httpusername -P httppassword
POA C/S Threads: 40 Total, 39 Idle, 97% Idle, Oldest 1 seconds

> sh check_gw_threads.sh -H ngwnameserver -r msg -W ./my-wgetrc
POA MsgWrk Threads: 6 Total, 6 Idle, 100% Idle, Oldest 0 seconds