These past few days I've been running some load tests on new servers, configurations and our Internet link. While we have a Cacti installation graphing everything via SNMP, the 5 minute polling interval means waiting for what seems like forever.

I just want some quick and clean bandwidth statistics!

After some years of needing a simple tool, finally bwm-ng has entered my life. Where have you been all these years, bwm-ng? I assume that bwm-ng is short for "bandwidth monitor, next generation."

My only complaint is that when monitoring network interfaces, ethernet bonds are treated as equals to the real interfaces rather than the virtual cumulative devices they are.

Other than that, this tool rocks. Some features that are most pleasing:

• Displays different units: bps to Mbps, Bps to MBps
• Network interfaces and hard drives
• Simple, clean and easy to comprehend user interface
• 100ms polling interval granularity
• apt-get installation on Debian Lenny

#geekloveatfirstsight

Add a comment

Posted by Stu Thompson on 30 July 2010 09:51:00 CEST #

While setting up a few Debian Lenny machines this summer, I came across the error message SIOCSIFADDR: No such device a few times. All of these servers have NIC bonding configured, and a few of them have multiple Ethernet bonds. Here are a couple of potential causes for this error message:

• The /etc/modprobe.d/arch/X86_64 file does not contain the bonded device name. For multiple bonded devices, the file must contain an alias entry for each. Here is an example for a two device system named bond0 and bond1:

  
  alias bond0 bonding
  alias bond1 bonding
  
  

• The /etc/modules module for bonding (bonding mode=4 miimon=100 max_bonds=2) is configured for fewer bonds than the server has. The here max_bonds=2 is the maximum number of bonding devices your system will have. The default is 1. If you machine has more, then SIOCSIFADDR: No such device will appear for the devices that did not come up.

Add a comment

Posted by Stu Thompson on 26 July 2010 11:19:00 CEST #

Just so I don't forget. Again.

berea:~# hpacucli 
HP Array Configuration Utility CLI 8.28-13.0
Detecting Controllers...Done.
Type "help" for a list of supported commands.
Type "exit" to close the console.

=> ctrl all show

Smart Array P400 in Slot 1                (sn: P61620D9SUK034)

=> ctrl slot=1 show

Smart Array P400 in Slot 1
   Bus Interface: PCI
   Slot: 1
   Serial Number: P61620D9SUK034
   Cache Serial Number: PA82C0H9SUJL94
   RAID 6 (ADG) Status: Disabled
   Controller Status: OK
   Chassis Slot: 
   Hardware Revision: Rev D
   Firmware Version: 7.08
   Rebuild Priority: Medium
   Expand Priority: Medium
   Surface Scan Delay: 15 secs
   Post Prompt Timeout: 0 secs
   Cache Board Present: True
   Cache Status: OK
   Accelerator Ratio: 100% Read / 0% Write
   Drive Write Cache: Disabled
   Total Cache Size: 256 MB
   No-Battery Write Cache: Disabled
   Battery/Capacitor Count: 0
   SATA NCQ Supported: True

=> ctrl slot=1 array all show

Smart Array P400 in Slot 1

   array A (SAS, Unused Space: 0 MB)
   array B (SAS, Unused Space: 0 MB)

=> ctrl slot=1 ld all show

Smart Array P400 in Slot 1

   array A

      logicaldrive 1 (68.3 GB, RAID 1, OK)

   array B

      logicaldrive 2 (136.7 GB, RAID 5, OK)

=> ctrl slot=1 pd all show

Smart Array P400 in Slot 1

   array A

      physicaldrive 2I:1:1 (port 2I:box 1:bay 1, SAS, 72 GB, OK)
      physicaldrive 2I:1:2 (port 2I:box 1:bay 2, SAS, 72 GB, OK)

   array B

      physicaldrive 1I:1:6 (port 1I:box 1:bay 6, SAS, 72 GB, OK)
      physicaldrive 1I:1:7 (port 1I:box 1:bay 7, SAS, 72 GB, OK)
      physicaldrive 1I:1:8 (port 1I:box 1:bay 8, SAS, 72 GB, OK)

   unassigned

      physicaldrive 2I:1:3 (port 2I:box 1:bay 3, SAS, 73.5 GB, OK)

=> ctrl slot=1 array a add spares=2I:1:3
=> ctrl slot=1 pd all show

Smart Array P400 in Slot 1

   array A

      physicaldrive 2I:1:1 (port 2I:box 1:bay 1, SAS, 72 GB, OK)
      physicaldrive 2I:1:2 (port 2I:box 1:bay 2, SAS, 72 GB, OK)
      physicaldrive 2I:1:3 (port 2I:box 1:bay 3, SAS, 73.5 GB, OK, spare)

   array B

      physicaldrive 1I:1:6 (port 1I:box 1:bay 6, SAS, 72 GB, OK)
      physicaldrive 1I:1:7 (port 1I:box 1:bay 7, SAS, 72 GB, OK)
      physicaldrive 1I:1:8 (port 1I:box 1:bay 8, SAS, 72 GB, OK)

=> 
=> exit

Add a comment

Posted by Stu Thompson on 23 July 2010 16:35:00 CEST #

So you need a fat network connection on your Debian 5 server, but a 10 GoE infrastructure is not in your budget? No fear! Link Aggregation Control Protocol is here to save the day!

Here's what I did:

• Purchase a NIC capable of LACP with solid Linux driver support. I went with an Intel PRO/1000 PT Quad Port Server Adapter.
• Purchase a Switch capable of LACP. It should be more than capable of handeling the bandwidth. We opted for an HP ProCurve 2510G-24.
• Install ifenslave-2.6: apt-get install ifenslave-2.6
• Purchase some network cables. Cat 6 if you can, Cat 5e if not.
• Edit /etc/modules and add bonding mode=4 miimon=100 max_bonds=2. This will load the module at boot time in the future. The value of max_bonds is the number of bonding devices your system will have. The default is 1.
• Load the module, so we can proceed. modprobe bonding. There should be no errors. Confirm it is loaded with modprobe -l | grep bond
• Edit /etc/network/interfaces to look something like this:
  

  # The loopback network interface
  auto lo
  iface lo inet loopback
  
  auto  bond0
  iface bond0 inet static
      address     192.168.1.131
      gateway     192.168.1.1
      broadcast   192.168.1.255
      netmask     255.255.255.0
      up   /sbin/ifenslave    bond0 eth0 eth1 eth2 eth3
      down /sbin/ifenslave -d bond0 eth0 eth1 eth2 eth3
  
  #similar configuration for bond1
  

• Add to /etc/modprobe.d/arch/X86_64 something like this:
  

  
  alias bond0 bonding
  alias bond1 bonding
  

• Plug in the network cables
  
  
  
• Restart the network: /etc/init.d/network restart
• Configure the switch to use the four ports as a single trunk. Here's how for an HP ProCurve:

• Confirm that there are no errors
  1. The NIC and switch LEDs should be active
  2. dmesg will show what the kernal thinks of your new configuration
  3. ifconfig should show your bond0 interface up
  4. The interface should be pingable both internally and from other computer

You should be good to go at this point!

Add a comment

Posted by Stu Thompson on 20 May 2010 09:46:00 CEST #

While there is some information out there on the Intertubes covering how to implement 80-->8080 port on a Linux box, they mainly pertain to single instance servers. The below iptables-restore input snippet works just great for such a setup:

*nat
#valid for tomcat listen to everywhere
-A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8080

#-A PREROUTING -j LOG --log-prefix "iptables routing: "  --log-level 7

COMMIT

But this configuration fails if your tomcat is configured to listen to a specific IP, as in when sharing a single server with multiple web servers. (Note the proxyPort attribute. This is important too.)

    <Connector port="8080" 
               proxyPort="80" 
               address="2.3.4.5"
               protocol="HTTP/1.1"
             >

No fear! A slightly more verbose iptables configuration file will do the trick!

cat /etc/iptables.up.rules
*nat
-A PREROUTING --dst 2.3.4.5 -p tcp --dport 80 -j DNAT --to-destination 2.3.4.5:8080
-A PREROUTING --dst 2.3.4.6 -p tcp --dport 80 -j DNAT --to-destination 2.3.4.6:8080
-A PREROUTING --dst 2.3.4.7 -p tcp --dport 80 -j DNAT --to-destination 2.3.4.7:8080

#-A PREROUTING -j LOG --log-prefix "iptables routing: "  --log-level 7

COMMIT

Presto! I worked this out with a bit of experimenting and a blog post I found: "Non-root Tomcat on Port 80 on a VPS without Apache"

Add a comment

Posted by Stu Thompson on 20 May 2010 09:09:00 CEST #