Re: [vserver] Suddenly loosing network-interfaces on some hosts

From: Michael Hoffrath <m.hoffrath_at_clano-it.com>
Date: Tue 23 Nov 2010 - 13:51:39 GMT
Message-Id: <4D050DB5-66D7-4979-B270-D385DCDB90B5@clano-it.com>

> On Tue, Nov 23, 2010 at 12:47:05PM +0100, Michael Hoffrath wrote:
>> Hello,
>
>> Summary: Guests suddenly loosing Network-Interfaces.
>
>> Description:
>> we've noticed a problem on 2 of our hosts, not sure if this could be
>> related to linux-vserver.
>
>> The System hosting between 5-15 Linux-Vserver running with kernel
>> 2.6.26-2-vserver-amd64.
>
> known broken kernel (see wiki) don't use it if you don't
> want to live with the known and unknown problems

We will upgrade the machines and keep them in view, thank you.

>
>> Both Systems suddenly through out this error (kernel.log):
>> Nov 22 11:41:54 kernel: [63954.956587] ------------[ cut here ]------------
>> Nov 22 11:41:54 kernel: [63954.956587] WARNING: at net/core/dst.c:265 dst_release+0x23/0x2c()
>> Nov 22 11:41:54 kernel: [63954.956587] Modules linked in: xt_multiport iptable_filter ip_tables x_tables binfmt_misc loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw psmouse button pcspkr evdev dcdbas ext3 jbd mbcache dm_mirror dm_log dm_snapshot dm_mod raid1 md_mod sg sr_mod cdrom sd_mod ide_pci_generic ide_core ata_generic ata_piix libata scsi_mod dock ehci_hcd uhci_hcd tg3 thermal processor fan thermal_sys [last unloaded: scsi_wait_scan]
>> Nov 22 11:41:54 kernel: [63954.956587] Pid: 16786, comm: java Tainted: G W 2.6.26-2-vserver-amd64 #1
>> Nov 22 11:41:54 kernel: [63954.956587]
>> Nov 22 11:41:54 kernel: [63954.956587] Call Trace:
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80234e68>] warn_on_slowpath+0x51/0x7a
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803e65c0>] __ip_route_output_key+0x1cf/0x8f8
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803e6d02>] ip_route_output_flow+0x19/0x1de
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803ce4c4>] dst_release+0x23/0x2c
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff804096a0>] udp_sendmsg+0x4fc/0x5d3
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803bf69d>] sock_sendmsg+0xed/0x1da
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80246f91>] autoremove_wake_function+0x0/0x2e
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80259c6f>] futex_wake+0x74/0xfa
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff8025a940>] do_futex+0xa6/0x844
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff803c0272>] sys_sendto+0xf3/0x127
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80212507>] read_tsc+0x9/0x20
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80254d7a>] getnstimeofday+0x39/0x98
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff80249b32>] ktime_get_ts+0x22/0x4b
>> Nov 22 11:41:54 kernel: [63954.956587] [<ffffffff8020beda>] system_call_after_swapgs+0x8a/0x8f
>> Nov 22 11:41:54 kernel: [63954.956587]
>> Nov 22 11:41:54 kernel: [63954.956587] ---[ end trace 9e9501ed44896e2c ]---
>
> looks unrelated at first glance, but might fall into the
> unknown brokenness category ...
>
>> And some of the Guests lost their network-interface. After a restart
>> of the guest the network interface comes back online, but this problem
>> occurred several times.
>
> more likely that you have a primary/secondaries issue
> here ...
>
>> Here are some more information:
>>
>> sh testme.sh
>> Linux-VServer Test [V0.17] Copyright (C) 2003-2006 H.Poetzl
>> chcontext is working.
>> chbind is working.
>> Linux 2.6.26-2-vserver-amd64 #1 SMP Thu Sep 16 16:20:47 UTC 2010 x86_64
>> Ea 0.30.216 236/glibc (DSa) <v13,net,v21,v22,v23,netv2>
> ~~~~~~~~
> there is no 0.30.216 release (yet)

Thats the unmodified output of this script!
>
> best luck,
> Herbert
>
>> VCI: 0002:0303 236 13000ff1 (KtTbsPHIiW)
>> ---
>> [000]# succeeded.
>> [001]# succeeded.
>> [011]# succeeded.
>> [031]# succeeded.
>> [101]# succeeded.
>> [102]# succeeded.
>> [201]# succeeded.
>> [202]# succeeded.
>>
>> ./testfs.sh -t -x -y -z -D /dev/loop0 -M /mnt
>> Linux-VServer FS Test [V0.23] Copyright (C) 2005-2009 H.Poetzl
>> Linux 2.6.26-2-vserver-amd64 x86_64/0.30.216
>> VCI: 0002:0303 236 13000ff1 (ID24)
>> ---
>> testing ext2 filesystem ...
>> [000].
>>
>> [001]. [002]. (ext2 format)
>> tag related tests ...
>> [011]. [012]. [014]. [015]. [019].
>> [020]. [021]. [022]. [023]. [024]. [025]. [026]. [027]. [028].
>> [033]. [034]. [035]. [037]. [045]. [047].
>> xattr related tests ...
>> [101]. [103]. [104]. [106]. [107]. [109].
>> [112]. [113]. [114]. [115]. [116]* [118]. [119].
>> [122]. [123]. [124]. [125]. [127]. [128]. [129].
>> [131]. [132]. [133]. [134]. [135]. [138]. [139].
>> [148]. [149].
>> disk limit related tests ...
>> [201]. [202]. [203]. [204]. [205]. [206]. [207]. [208].
>> [211]. [212]. [213]. [222]. [223]. [231]. [232]. [233]. [239].
>> [999].
>> ---
>> testing ext3 filesystem ...
>> [000]. [001]. [002]. (ext3 format)
>> tag related tests ...
>> [011]. [012]. [014]. [015]. [019].
>> [020]. [021]. [022]. [023]. [024]. [025]. [026]. [027]. [028].
>> [033]. [034]. [035]. [037]. [045]. [047].
>> xattr related tests ...
>> [101]. [103]. [104]. [106]. [107]. [109].
>> [112]. [113]. [114]. [115]. [116]* [118]. [119].
>> [122]. [123]. [124]. [125]. [127]. [128]. [129].
>> [131]. [132]. [133]. [134]. [135]. [138]. [139].
>> [148]. [149].
>> disk limit related tests ...
>> [201]. [202]. [203]. [204]. [205]. [206]. [207]. [208].
>> [211]. [212]. [213]. [222]. [223]. [231]. [232]. [233]. [239].
>> [999].
>> ---
>> testing ext4 filesystem ...
>> [000]. [001]*
>> ---
>> testing xfs filesystem ...
>> [000]* (xfs format failed)
>> ---
>> testing reiser filesystem ...
>> [000]* (reiserfs format failed)
>> ---
>> testing jfs filesystem ...
>> [000]* (jfs format failed)
>>
>> Kernel:
>> Linux version 2.6.26-2-vserver-amd64 (Debian 2.6.26-25lenny1) (dannf@debian.org) (gcc version 4.1.3 20080704 (prerelease) (Debian 4.1.2-25)) #1 SMP Thu Sep 16 16:20:47 UTC 2010
>>
>> Processor:
>> processor : 3
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 23
>> model name : Intel(R) Xeon(R) CPU X3330 @ 2.66GHz
>> stepping : 10
>> cpu MHz : 2666.859
>> cache size : 3072 KB
>> physical id : 0
>> siblings : 4
>> core id : 3
>> cpu cores : 4
>> apicid : 3
>> initial apicid : 3
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 13
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
>> bogomips : 5333.77
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 36 bits physical, 48 bits virtual
>> power management:
>>
>> Modules:
>> ext2 68880 0 - Live 0xffffffffa01e6000
>> xt_multiport 7424 1 - Live 0xffffffffa01e3000
>> iptable_filter 7424 1 - Live 0xffffffffa01e0000
>> ip_tables 21520 1 iptable_filter, Live 0xffffffffa01d9000
>> x_tables 25224 2 xt_multiport,ip_tables, Live 0xffffffffa0195000
>> binfmt_misc 13580 1 - Live 0xffffffffa0190000
>> loop 19596 1 - Live 0xffffffffa018a000
>> snd_pcm 81800 0 - Live 0xffffffffa01c4000
>> snd_timer 25744 1 snd_pcm, Live 0xffffffffa01bc000
>> snd 63688 2 snd_pcm,snd_timer, Live 0xffffffffa01ab000
>> soundcore 12064 1 snd, Live 0xffffffffa01a7000
>> snd_page_alloc 13072 1 snd_pcm, Live 0xffffffffa01a2000
>> serio_raw 9988 0 - Live 0xffffffffa019e000
>> psmouse 42268 0 - Live 0xffffffffa017e000
>> button 11680 0 - Live 0xffffffffa017a000
>> pcspkr 7040 0 - Live 0xffffffffa0177000
>> evdev 14208 0 - Live 0xffffffffa0172000
>> dcdbas 11952 0 - Live 0xffffffffa016e000
>> ext3 127632 28 - Live 0xffffffffa014d000
>> jbd 51240 1 ext3, Live 0xffffffffa013f000
>> mbcache 12804 2 ext2,ext3, Live 0xffffffffa013a000
>> dm_mirror 20608 0 - Live 0xffffffffa0133000
>> dm_log 13956 1 dm_mirror, Live 0xffffffffa012e000
>> dm_snapshot 19400 0 - Live 0xffffffffa0128000
>> dm_mod 59376 56 dm_mirror,dm_log,dm_snapshot, Live 0xffffffffa0118000
>> raid1 24192 4 - Live 0xffffffffa0111000
>> md_mod 80292 5 raid1, Live 0xffffffffa00a6000
>> sg 36448 0 - Live 0xffffffffa0107000
>> sr_mod 19652 0 - Live 0xffffffffa0101000
>> cdrom 37928 1 sr_mod, Live 0xffffffffa00f6000
>> sd_mod 29376 10 - Live 0xffffffffa00ed000
>> ide_pci_generic 9220 0 [permanent], Live 0xffffffffa00e9000
>> ide_core 128284 1 ide_pci_generic, Live 0xffffffffa00c8000
>> ata_generic 10116 0 - Live 0xffffffffa00c4000
>> ata_piix 22916 8 - Live 0xffffffffa00bd000
>> libata 165600 2 ata_generic,ata_piix, Live 0xffffffffa007c000
>> scsi_mod 161016 4 sg,sr_mod,sd_mod,libata, Live 0xffffffffa0053000
>> dock 14112 1 libata, Live 0xffffffffa004e000
>> ehci_hcd 36108 0 - Live 0xffffffffa0042000
>> uhci_hcd 25760 0 - Live 0xffffffffa0038000
>> tg3 97156 0 - Live 0xffffffffa001d000
>> thermal 22688 0 - Live 0xffffffffa0016000
>> processor 42304 1 thermal, Live 0xffffffffa000a000
>> fan 9352 0 - Live 0xffffffffa0006000
>> thermal_sys 17728 3 thermal,processor,fan, Live 0xffffffffa0000000
>>
>> Kind Regards,
>> Michael
Received on Tue Nov 23 13:52:25 2010

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Tue 23 Nov 2010 - 13:52:25 GMT by hypermail 2.1.8