[vserver] Suddenly loosing network-interfaces on some hosts

From: Michael Hoffrath <m.hoffrath_at_clano-it.com>
Date: Tue 23 Nov 2010 - 11:47:05 GMT
Message-Id: <74688F82-6D20-4274-A5FD-331BD56EA448@clano-it.com>

Hello,

Summary: Guests suddenly loosing Network-Interfaces.

Description:
we've noticed a problem on 2 of our hosts, not sure if this could be related to linux-vserver.

The System hosting between 5-15 Linux-Vserver running with kernel 2.6.26-2-vserver-amd64.

Both Systems suddenly through out this error (kernel.log):
Nov 22 11:41:54 kernel: [63954.956587] ------------[ cut here ]------------
Nov 22 11:41:54 kernel: [63954.956587] WARNING: at net/core/dst.c:265 dst_release+0x23/0x2c()
Nov 22 11:41:54 kernel: [63954.956587] Modules linked in: xt_multiport iptable_filter ip_tables x_tables binfmt_misc loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw psmouse button pcspkr evdev dcdbas ext3 jbd mbcache dm_mirror dm_log dm_snapshot dm_mod raid1 md_mod sg sr_mod cdrom sd_mod ide_pci_generic ide_core ata_generic ata_piix libata scsi_mod dock ehci_hcd uhci_hcd tg3 thermal processor fan thermal_sys [last unloaded: scsi_wait_scan]
Nov 22 11:41:54 kernel: [63954.956587] Pid: 16786, comm: java Tainted: G W 2.6.26-2-vserver-amd64 #1
Nov 22 11:41:54 kernel: [63954.956587]
Nov 22 11:41:54 kernel: [63954.956587] Call Trace:
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80234e68>] warn_on_slowpath+0x51/0x7a
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803e65c0>] __ip_route_output_key+0x1cf/0x8f8
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803e6d02>] ip_route_output_flow+0x19/0x1de
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803ce4c4>] dst_release+0x23/0x2c
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff804096a0>] udp_sendmsg+0x4fc/0x5d3
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803bf69d>] sock_sendmsg+0xed/0x1da
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80246f91>] autoremove_wake_function+0x0/0x2e
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80259c6f>] futex_wake+0x74/0xfa
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff8025a940>] do_futex+0xa6/0x844
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803c0272>] sys_sendto+0xf3/0x127
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80212507>] read_tsc+0x9/0x20
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80254d7a>] getnstimeofday+0x39/0x98
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80249b32>] ktime_get_ts+0x22/0x4b
Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff8020beda>] system_call_after_swapgs+0x8a/0x8f
Nov 22 11:41:54 kernel: [63954.956587]
Nov 22 11:41:54 kernel: [63954.956587] ---[ end trace 9e9501ed44896e2c ]---

And some of the Guests lost their network-interface. After a restart of the guest the network interface comes back online, but this problem occurred several times.

Here are some more information:

sh testme.sh
Linux-VServer Test [V0.17] Copyright (C) 2003-2006 H.Poetzl
chcontext is working.
chbind is working.
Linux 2.6.26-2-vserver-amd64 #1 SMP Thu Sep 16 16:20:47 UTC 2010 x86_64
Ea 0.30.216 236/glibc (DSa) <v13,net,v21,v22,v23,netv2>
VCI: 0002:0303 236 13000ff1 (KtTbsPHIiW)

---
[000]# succeeded.
[001]# succeeded.
[011]# succeeded.
[031]# succeeded.
[101]# succeeded.
[102]# succeeded.
[201]# succeeded.
[202]# succeeded.
./testfs.sh -t -x -y -z -D /dev/loop0 -M /mnt
Linux-VServer FS Test [V0.23] Copyright (C) 2005-2009 H.Poetzl
Linux 2.6.26-2-vserver-amd64 x86_64/0.30.216
VCI:  0002:0303 236 13000ff1 (ID24)
---
testing ext2 filesystem ...
[000]. 
[001]. [002]. (ext2 format)
tag related tests ...
[011]. [012]. [014]. [015]. [019]. 
[020]. [021]. [022]. [023]. [024]. [025]. [026]. [027]. [028]. 
[033]. [034]. [035]. [037]. [045]. [047]. 
xattr related tests ...
[101]. [103]. [104]. [106]. [107]. [109]. 
[112]. [113]. [114]. [115]. [116]* [118]. [119]. 
[122]. [123]. [124]. [125]. [127]. [128]. [129]. 
[131]. [132]. [133]. [134]. [135]. [138]. [139]. 
[148]. [149]. 
disk limit related tests ...
[201]. [202]. [203]. [204]. [205]. [206]. [207]. [208]. 
[211]. [212]. [213]. [222]. [223]. [231]. [232]. [233]. [239]. 
[999]. 
---
testing ext3 filesystem ...
[000]. [001]. [002]. (ext3 format)
tag related tests ...
[011]. [012]. [014]. [015]. [019]. 
[020]. [021]. [022]. [023]. [024]. [025]. [026]. [027]. [028]. 
[033]. [034]. [035]. [037]. [045]. [047]. 
xattr related tests ...
[101]. [103]. [104]. [106]. [107]. [109]. 
[112]. [113]. [114]. [115]. [116]* [118]. [119]. 
[122]. [123]. [124]. [125]. [127]. [128]. [129]. 
[131]. [132]. [133]. [134]. [135]. [138]. [139]. 
[148]. [149]. 
disk limit related tests ...
[201]. [202]. [203]. [204]. [205]. [206]. [207]. [208]. 
[211]. [212]. [213]. [222]. [223]. [231]. [232]. [233]. [239]. 
[999]. 
---
testing ext4 filesystem ...
[000]. [001]* 
---
testing xfs filesystem ...
[000]* (xfs format failed)
---
testing reiser filesystem ...
[000]* (reiserfs format failed)
---
testing jfs filesystem ...
[000]* (jfs format failed)
Kernel:
Linux version 2.6.26-2-vserver-amd64 (Debian 2.6.26-25lenny1) (dannf@debian.org) (gcc version 4.1.3 20080704 (prerelease) (Debian 4.1.2-25)) #1 SMP Thu Sep 16 16:20:47 UTC 2010
Processor:
processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Xeon(R) CPU           X3330  @ 2.66GHz
stepping        : 10
cpu MHz         : 2666.859
cache size      : 3072 KB
physical id     : 0
siblings        : 4
core id         : 3
cpu cores       : 4
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
bogomips        : 5333.77
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:
Modules:
ext2 68880 0 - Live 0xffffffffa01e6000
xt_multiport 7424 1 - Live 0xffffffffa01e3000
iptable_filter 7424 1 - Live 0xffffffffa01e0000
ip_tables 21520 1 iptable_filter, Live 0xffffffffa01d9000
x_tables 25224 2 xt_multiport,ip_tables, Live 0xffffffffa0195000
binfmt_misc 13580 1 - Live 0xffffffffa0190000
loop 19596 1 - Live 0xffffffffa018a000
snd_pcm 81800 0 - Live 0xffffffffa01c4000
snd_timer 25744 1 snd_pcm, Live 0xffffffffa01bc000
snd 63688 2 snd_pcm,snd_timer, Live 0xffffffffa01ab000
soundcore 12064 1 snd, Live 0xffffffffa01a7000
snd_page_alloc 13072 1 snd_pcm, Live 0xffffffffa01a2000
serio_raw 9988 0 - Live 0xffffffffa019e000
psmouse 42268 0 - Live 0xffffffffa017e000
button 11680 0 - Live 0xffffffffa017a000
pcspkr 7040 0 - Live 0xffffffffa0177000
evdev 14208 0 - Live 0xffffffffa0172000
dcdbas 11952 0 - Live 0xffffffffa016e000
ext3 127632 28 - Live 0xffffffffa014d000
jbd 51240 1 ext3, Live 0xffffffffa013f000
mbcache 12804 2 ext2,ext3, Live 0xffffffffa013a000
dm_mirror 20608 0 - Live 0xffffffffa0133000
dm_log 13956 1 dm_mirror, Live 0xffffffffa012e000
dm_snapshot 19400 0 - Live 0xffffffffa0128000
dm_mod 59376 56 dm_mirror,dm_log,dm_snapshot, Live 0xffffffffa0118000
raid1 24192 4 - Live 0xffffffffa0111000
md_mod 80292 5 raid1, Live 0xffffffffa00a6000
sg 36448 0 - Live 0xffffffffa0107000
sr_mod 19652 0 - Live 0xffffffffa0101000
cdrom 37928 1 sr_mod, Live 0xffffffffa00f6000
sd_mod 29376 10 - Live 0xffffffffa00ed000
ide_pci_generic 9220 0 [permanent], Live 0xffffffffa00e9000
ide_core 128284 1 ide_pci_generic, Live 0xffffffffa00c8000
ata_generic 10116 0 - Live 0xffffffffa00c4000
ata_piix 22916 8 - Live 0xffffffffa00bd000
libata 165600 2 ata_generic,ata_piix, Live 0xffffffffa007c000
scsi_mod 161016 4 sg,sr_mod,sd_mod,libata, Live 0xffffffffa0053000
dock 14112 1 libata, Live 0xffffffffa004e000
ehci_hcd 36108 0 - Live 0xffffffffa0042000
uhci_hcd 25760 0 - Live 0xffffffffa0038000
tg3 97156 0 - Live 0xffffffffa001d000
thermal 22688 0 - Live 0xffffffffa0016000
processor 42304 1 thermal, Live 0xffffffffa000a000
fan 9352 0 - Live 0xffffffffa0006000
thermal_sys 17728 3 thermal,processor,fan, Live 0xffffffffa0000000
Kind Regards,
Michael
Received on Tue Nov 23 11:47:50 2010
[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Tue 23 Nov 2010 - 11:47:50 GMT by hypermail 2.1.8