Basic AIX network monitoring

Ø netstat –v | grep -i "No Resource Errors:"

o “No Resource Errors” can occur when the appropriate amount of memory can not be added quickly to vent buffer space for a workload situation. If “No Resource Errors” continues increasing, it means mbuf values are not sufficient and must be increased.

Ø # netstat -v | grep Queue

Max Packets on S/W Transmit Queue: 0

S/W Transmit Queue Overflow: 0

Current S/W+H/W Transmit Queue Length: 0

Current HW Transmit Queue Length: 0

o If "Max packets on S/W transmit queue" is >0 and is equal to current "HW transmit queue length" then the "send queue size" should be increased.

Ø # netstat -v | grep mbuf

No mbuf Errors: 0

o If the "No mbuf errors" is large then the "receive queue size" needs to be increased.

Ø Take care about these values

netstat -p tcp|grep -i "Connections dropped due to bad ACKs"
netstat -p tcp|grep -i "discarded by listeners"
netstat –v | grep -i "Packets Dropped:"
netstat –v | grep -i "Hypervisor Receive Failures:"

Ø Look at queue values here.

[root@server1]</>netstat -ano | more
tcp4 0 0 *.80 *.* LISTEN

so_options: (ACCEPTCONN|REUSEADDR|REUSEPORT|AUDIT)
q0len:0 qlen:0 qlimit:50 so_state: ()
timeo:0 uid:400

Ø netstat -i

Both Oerrs (number of output errors since boot) and Ierrs (Input errors since boot) should be < 0.025.
If Oerrs>0.025 then it is worth increasing the send queue size.
Ierrs includes checksum errors and can also be an indicator of a hardware error such as a bad connector or terminator.

The Collis field shows the number of collisions since boot and can be as high as 10%.
If it is greater then it is necessary to reorganize the network as the network is obviously overloaded on that segment.

Ø Netstat –m

netstat -m s used to analyze the use of mbufs in order to determine whether these are the bottleneck.
The no -a command is used to see what the current values are. Values of interest are thewall, lowclust, lowmbuf and dogticks.

An mbuf is a kernel buffer that uses pinned memory and is used to service network communications.
Mbufs come in two sizes - 256 bytes and 4096 bytes (clusters of 256 bytes).

Thewall is the maximum memory that can be taken up for mbufs. Lowmbuf is the minimum number of mbufs to be
kept free while lowclust is the minimum number of clusters to be kept free. Mb_cl_hiwat is the maximum number of free buffers to be kept in the free buffer pool and should be set to at least twice the value of lowclust to avoid thrashing.

NB by default AIX sets thewall to half of memory which should be plenty. It is now a restricted tunable.
# no -a -F | grep thewall
thewall = 1572864

# vmstat 1 1
System configuration: lcpu=4 mem=3072MB ent=0.50

*** Please feel free to communicate by bulent.yucesoy@gmail.com