Solve the increased CPU and TIME_WAIT connections in LAG

0 Likes

Problem



The Linux Access Gateway (LAG)  increases CPU and TIME_WAIT connections when it is stressed.

When the LAG has too many requests, activity rises on the ics_dyn process, which is reflected in a somewhat higher CPU consumption and increased connections TIME_WAIT.

Such connections are transient, but having many in memory can cause degradation in the server.

Solution



Enable flags using the product to manage these connections and optimize TCP connections through tuning. (According to the documentation:)

In the script edit the following values:


  • Decrease the time default value for tcp_fin_timeout connection.

  • Decrease the time default value for tcp_keepalive_time connection.



<- HARDCORE Stuff! ->


  • Increase the length of the processor input queue.

  • Enable recycle and reuse connections.



This script was tested in: Novell Access Manager 3.1 SP4 Linux Access Gateway

With excellent and fast results.

PLEASE USE THIS SCRIPT WITH FULL KNOWLEDGE AND AT YOUR OWN RISK

#!/bin/bash
# =====================================================
# ics_dyn:
# Solve the high CPU usage setting flags to the NAM
# TIME_WAIT:
# Stabilizes connections with a TCP tunning
# =====================================================
# William Vera wvera@novell.com
# Feb 21 2013 V0.002
# =====================================================

# ics_dyn
if [ -f /var/opt/novell/naudit/nproduct.log ];then rm /var/opt/novell/naudit/nproduct.log;fi
touch /var/novell/.releaseclosewait
touch /var/novell/.fixCloseWait
touch /var/novell/.releasetimedoutbrowserconn
/etc/init.d/novell-vmc stop
rm /var/novell/.~newInstall
/etc/init.d/novell-vmc start

cat >> /etc/sysctl.conf << EOF
# TCP Tunning - DON'T EDIT BELOW
net.ipv4.tcp_fin_timeout = 25
net.core.netdev_max_backlog = 2500
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
# Detect network errores
net.ipv4.tcp_keepalive_time=60
net.ipv4.tcp_keepalive_probes=3
net.ipv4.tcp_keepalive_intvl=15
EOF
/sbin/sysctl -p >/dev/null 2>&1
/sbin/sysctl -w net.ipv4.route.flush=1 >/dev/null 2>&1


Almost immediately after running the script, you may notice a change in CPU use and connections TIME_WAIT.

You can check with the command below before and after running the script to see the difference.
netstat -nap | awk '/tcp/ {print $6}'| sort | uniq -c

Labels:

How To-Best Practice
Collateral
Comment List
Anonymous
Related Discussions
Recommended