[00:29] shuri (~ipv6@cpu183.adsl.qc.bellglobal.com) joined #vserver. [00:59] Doener (~doener@pD9E12458.dip.t-dialin.net) joined #vserver. [00:59] Nick change: Bertl_zZ -> Bertl_oO [01:00] Bertl_oO: what does _oO mean [01:00] Out of Ofice/Order ... [01:01] It means, I'm awake, but I'm not really there ;) [01:02] okay [01:36] force_ (force@brln-d9ba1fed.pool.mediaWays.net) joined #vserver. [01:37] hello! is somebody here who could perhaps help me? [01:37] i have a really weird problem! [01:37] what is the problem ? [01:38] one sec, ill upload the log [01:39] ok, following situation [01:39] i have a server running with vserver1.22 and kernel 2.4.23 [01:40] its a AMD2800+ machine with 1,5GB ram [01:40] and there are about 10 vservers running on it [01:40] now what happens every about 1-2 days is that the machine crashes [01:41] or better it doesnt crash, but nobody can access it anymore [01:41] its seems to be fully loaded [01:41] but i can see that it it still running because the shoutcast streams on the machine still run (but very laggy) [01:42] now since i thought this could be because one vserver is going wild, i set on all vservers hardlimits with ulimit [01:42] after this something strange happened after about 1 day [01:42] have you tried setting the sched flag ? [01:42] what happened ? [01:44] i couldnt access the machine at all, but the users which where still on the machine, reported that everything they tried to start, crashed with a segfault [01:44] now i again had to reboot the machine [01:45] through hardware reset [01:45] after the previous reboots, i didnt find anything in the logs.. at least nothing suspicious [01:45] but this time [01:45] ( i think cause i set the hardlimits) [01:45] the logs where full [01:46] this is what was in the kern.log: [01:46] http://www.traffic4all.com/ksymoops.txt [01:46] (i ran it through the ksymoops program [01:46] JonB could u possibly know what is wrong with my machine? [01:48] i'm not smart enough for ksymoops, you need Bertl_oO for that [01:48] have you tried setting the sched flag ? [01:48] did you test the memory? [01:48] does it work with a normal kernel ? [01:48] xsbyme (xsbyme@D5E0EDEB.kabel.telenet.be) joined #vserver. [01:49] force_: is the cpu overclocked? [01:49] hm, well we tested not thoroughly.. [01:49] no cpu isnt overclocked [01:49] force_: how hot is the cpu ? [01:49] i dont know if it runs stable with the standard kernel [01:49] i also cant tell that, i tried to get lmsensors running, but it didnt work [01:50] but do u think this could be a hardware problem? [01:50] oh i forgot to tell, the system the server is running is gentoo [01:50] and the vservers are running mainly debian or redhat [01:50] force_ some friends say se think its in the software [01:50] xsbyme hm, yes but what exactly? ;) [01:51] well, i think software, but i want to rule out the hardware [01:51] JonB what is the sched flag? [01:52] force_: the sched flag makes all processes in one context scheduled as one [01:52] hm, what does this mean? or where do i find documentation on this? :) [01:52] force_ tozz dont he only know that is the software [01:54] force_: on the website [01:54] force_: maybe the original from jacques [01:54] ok, thx, ill take a look [01:54] ccooke (~ccooke@80.1.164.238) left irc: Ping timeout: 499 seconds [02:03] force_ [02:03] http://www.gnu.org/directory/devel/debug/ksymoops.html [02:07] JonB do you possibly have a fairly good vserver config file? [02:08] Doener (~doener@pD9E12458.dip.t-dialin.net) left irc: Quit: Leaving [02:09] Doener (~doener@pD9E12458.dip.t-dialin.net) joined #vserver. [02:11] force_: what do you need ? [02:12] force_: which userland tools are you using ? [02:12] hm, does which can be found at 13thfloor.at [02:12] those [02:12] util-vserver ? [02:13] hm, util-vserver0.26 and vserver-0.29 [02:15] made by 2 different people [02:15] and they do the same [02:15] argh.. oh [02:15] anyway, what is it you want in your .config file ? [02:15] well i just need some good reference.. [02:16] hm, ill upload my config of a vserver and u tell me if it is ok? [02:17] well, mine is just, S_HOSTNAME, IPROOT, ONBOOT, S_NICE (empty), S_FLAGS="lock nproc", ULIMIT [02:17] i use the util-vserver [02:18] www.traffic4all.com/nihal.conf [02:18] hm, well what is better [02:18] util-server or the other tool?` [02:18] (the sched i just added.. didnt have that on bevore) [02:21] force_: just a sec [02:21] thx [02:22] i think it looks fine [02:23] k [02:33] JonB btw, is it normal, that when i use vtop, i see all processes but the processes of the vservers run e.g. as user XFS oder user #51 and so on!? [02:36] i havent used vtop [02:36] oh k [02:36] ah right u use the other tools [02:36] is there any difference? [02:36] no, i'm the only user of my server [02:37] oh ok :) [02:38] and it's just an old old p200mmx [02:38] heh, k i see :) [02:39] when i get my laptop (hopefully monday after 17, i will make my current desktop my server [02:49] JonB (~jon@129.142.112.33) left irc: Quit: zzzzz [03:39] force_ (force@brln-d9ba1fed.pool.mediaWays.net) left irc: Quit: Der Mensch wird vom Geist geleitet. In der Wüste bin ich das wert, was meine Gottheiten wert sind. [03:54] xsbyme (xsbyme@D5E0EDEB.kabel.telenet.be) left irc: [03:54] xsbyme (xsbyme@D5E0EDEB.kabel.telenet.be) joined #vserver. [04:26] ccooke (~ccooke@80.1.164.238) joined #vserver. [04:53] hm how did you make/get that oops logfile/ [05:29] ccooke (~ccooke@80.1.164.238) left irc: Ping timeout: 499 seconds [05:36] shuri (~ipv6@cpu183.adsl.qc.bellglobal.com) left irc: Ping timeout: 499 seconds [05:39] mcp (~hightower@wolk-project.de) joined #vserver. [07:04] James (~James@ip68-96-180-27.lv.lv.cox.net) joined #vserver. [07:04] allo all? [07:44] Tamama (~bgbgbg@a62-216-20-152.adsl.cistron.nl) left irc: Read error: Connection reset by peer [07:59] mdaur__ (mdaur@p509160A9.dip.t-dialin.net) joined #vserver. [08:04] mdaur_ (mdaur@pD9E05E8E.dip.t-dialin.net) left irc: Ping timeout: 485 seconds [08:17] James (~James@ip68-96-180-27.lv.lv.cox.net) left irc: Quit: HydraIRC -> http://www.hydrairc.com <- s0 d4Mn l33t |t'z 5c4rY! [10:11] loger joined #vserver. [11:14] serving (~serving@213.186.190.30) left irc: Ping timeout: 499 seconds [11:29] serving (~serving@213.186.190.72) joined #vserver. [11:48] JonB (~jon@129.142.112.33) joined #vserver. [13:27] JonB (~jon@129.142.112.33) left irc: Quit: Client exiting [14:33] loger joined #vserver. [15:51] Tamama (~bgbgbg@a62-216-20-152.adsl.cistron.nl) joined #vserver. [16:11] JonB (~jon@129.142.112.33) joined #vserver. [16:12] Nihal (force@brln-d9ba1fed.pool.mediaWays.net) joined #vserver. [16:12] hello! is someone available and can help? [16:12] :) [16:13] lo [16:14] Nihal whats rong [16:14] i have never seen 'someone' [16:15] i never saw anyone asking for 'can' help [16:15] :> [16:15] well, are there known problems with gentoo distro and vserver? [16:15] well ok then ill just ask [16:15] JonB u know my problem! ;) [16:15] Nihal: i do ? [16:15] Nihal: what is it ? [16:15] if you can't open a can, apply hammer [16:15] heh [16:15] JonB didtn we speak yesterday? [16:16] Nihal: not with that nick [16:16] i have a server running (amd2800+, 1,5GB ram) with about 10 vserver [16:16] oh [16:16] Nihal: that does sound awfull familiar [16:16] perhaps i was called force_ [16:16] ;) [16:16] yes [16:16] yeah that rings a bell [16:16] well, it just crashed again [16:17] but now after reboot all passwords of the vservers seem to have changed! [16:17] lol [16:17] at least i cant login with ssh in any vserver, always saying password wrong :( [16:17] are you sure that is not a hardware problem? [16:17] i _really_ dont know what it is [16:18] although i just found this in the logs: [16:18] we had a server going 'poop' on us as well.. just replaced a harddisk and it was all good.. [16:18] Dec 21 11:05:52 t4a3 kernel: EXT2-fs error (device lvm(58,5)): ext2_new_block: Free blocks count corrupted for block group 10 [16:18] Dec 21 11:12:55 t4a3 kernel: EXT2-fs error (device lvm(58,5)): ext2_new_block: Free blocks count corrupted for block group 10 [16:18] heh [16:18] Nihal: running raid ? [16:18] could it really be hardware? [16:18] JonB no.. just one harddisk, but we are using lvm [16:19] i think you can say : fubar [16:19] Nihal: no raid? BAAAAAD idea [16:20] JonB why? :) [16:20] Tamama hm? [16:20] Nihal: what happens if the disk dies ? [16:21] JonB yes, ok, thats an argument.. :) [16:21] but well thats currentyl secundary, it doesnt matter if it dies, if the machine doesnt run at all stable [16:21] i just cant figure out whats wrong [16:21] Nihal: suppose you had 2 disks, and one of them is semi bad [16:22] Tamama www.traffic4all.com/ksymoops.txt .. this are msgs i got yesterday, i ran it throzugh ksymoops .. could u see anything which could be the fault?= [16:22] JonB yes, i think we are going to replace the harddisc [16:23] Nihal: get 2 disks, and run raid, at least software raid [16:23] yes [16:23] going to do that :) [16:26] we stopped using that.. we just use 3ware raid controllers nad raid 5 now [16:27] Tamama: why ? [16:27] sure costs a bit more, but saves a lot of headaches [16:27] heh [16:27] which headaches ? [16:28] raid-1 only protects against 'poof harddisk dead' situations.. there is no fault tollerance so if 1 disk still works but is somewhat shakey, in the end you end up with 2 fucked up drives [16:28] thats not my experience [16:28] it is mine [16:29] i' not saying everybody should use raid5.. i just say i wont use anything less :) [16:30] hm, i just see that the controller [16:30] has an irq conflict [16:30] could this be the cause? [16:30] with what does it have a conflict then? [16:32] Tamama: i've experienced drives with bad sectors being detected by the linux software raid1 driver [16:33] Nihal: irq conflicts is a very very bad thing [16:33] jonb: well i have experienced a box going down and down and down ;) [16:33] i dont know, IRQ routing conflict for 00:0f.0, have irq 12, want irq 3 [16:33] that comes for the ide controller [16:33] dunno what it is conflicting with [16:34] what hardware do you have in your machine ? [16:35] Tamama: besides, software raid is faster and more configable. Eg. with 3ware you cant upgrade your disk drives and use the new space [16:35] its a KT400 board [16:35] dunno which one [16:35] did you disable onboard sound? [16:35] uuh.. software raid is NOT faster in my experience ;) [16:35] but i guess we have just opposite stands on this one [16:36] Tamama: well, i was thinking the same, but i benchmarked various 3ware controllers against software raid [16:36] raid 0 1 or 5? [16:36] Tamama: software raid won every time [16:37] Tamama: all of them [16:37] Tamama what you think of raid 6 and 7 just asking [16:38] fleshcraw (~fleshcraw@port-212-202-16-246.reverse.qsc.de) joined #vserver. [16:38] raid6 and 7 ? [16:38] Nick change: fleshcraw -> fleshcrawler [16:39] ye [16:39] s [16:39] what is that [16:41] Tamama: cant you recieve /dcc ? [16:41] nope [16:41] what 3ware controller did the benchmark test on? and what system? [16:41] JonB RAID 6: Independent Data disks with two independent distributed parity schemes [16:41] Tamama: read the .pdf [16:42] dont have it ne? heh [16:42] xsbyme: gives ? [16:42] Tamama: well, i did offer you a /dcc [16:42] and we just established i can not receive dccs [16:42] dont know them just know they exist [16:42] RAID 7: Optimized Asynchrony for High I/O Rates as well as High Data Transfer Rates [16:42] Tamama: do you have a mail address ? [16:43] just give me an weblink heh [16:43] i dont have that [17:03] Arvind-NL (~Hoi@ip503d4396.speed.planet.nl) joined #vserver. [17:03] hi [17:03] http://www.linux-vserver.org/index.php?page=Step-by-Step+Guide [17:03] I would like to know more about step 4 [17:03] what options do need to be enabled ? [17:04] kernel options ? [17:06] yes JonB [17:07] What options do I need to enable , i already patched the kernel [17:07] the usual, vserver has no specific kernel options [17:07] I have the ctx patch [17:07] kernel is already patched but to be honest the documentation is poor for a newbie like me [17:08] Arvind-NL: please elaborate to what you wish the documentation had included [17:09] xsbyme (xsbyme@D5E0EDEB.kabel.telenet.be) left irc: [17:09] xsbyme (xsbyme@D5E0EDEB.kabel.telenet.be) joined #vserver. [17:09] xsbyme (xsbyme@D5E0EDEB.kabel.telenet.be) left irc: Client Quit [17:09] A more verbose guide about configuring the kernel , I think that step 1 , 2 , 3 are clear but 4 could be more verbose i.m.o. [17:10] what options are made availble after patching the kernel [17:10] none [17:11] I see new ones in Block devices [17:11] <*> Virtual Root device support (NEW) [17:11] oh, then it has changed since i did mine [17:11] which patch are you using ? [17:11] patch-2.4.22-3-vs1.21.diff.gz [17:12] okay, i'm still at vs-1.0 [17:13] all of the 10 points are clear exept for step 4. hmmm [17:13] JonB> Could you provide me some more urls / docuementations about configuring vps with this patch set? [17:14] vps is a slightly different project [17:14] hm [17:14] i have a problem with the sshd of the vservers, i get a Dec 21 15:09:33 ip171 sshd[1586]: error: Bind to port 22 on 0.0.0.0 failed: Address already in use. [17:14] Dec 21 15:09:33 ip171 sshd[1586]: fatal: Cannot bind any address. [17:14] anyone know how to fix this? [17:15] Nihal: yes, make it bind to only it's own ip [17:15] hm, well it worked before that crash without setting the ip!? [17:15] Nihal: well, Address already in use [17:16] Arvind-NL: you need to be a little more specific about what you want help with [17:16] Tamama: still here ? [17:16] yes, but why does this suddenly appear? because before it worked.. like just bevore the crash! :( [17:17] Nihal: another daemon stole that port [17:17] JonB yes, the daemon from the main server is running there ... [17:17] Nihal: well, you cant run 2 daemons at the same port, now can you ? [17:17] can this be fixed without setting in every vserver the ip? [17:17] yes [17:18] yes [17:18] run sshd at a different port [17:19] k [17:19] well [17:19] i just made the main sshd run only on one ip :) [17:21] JonB> I would like to know what options I need to enable in the kernel config to use VPS. [17:21] Arvind-NL: vserver is NOT VPS [17:21] I'm browsing through the config now :) [17:24] Nick change: Bertl_oO -> Bertl [17:24] hi guys! [17:24] ahhh, Bertl, the guru :) [17:24] huh where? [17:25] Action: JonB points ---------> at Bertl and says "there" [17:25] Action: Bertl should have chosen some other nick ;) [17:25] okay what's the issue? [17:26] Bertl: Nihal has a crash issue, and Arvind-NL wants to know which kernel options to enable with patch-2.4.22-3-vs1.21.diff.gz [17:27] Bertl: www.traffic4all.com/ksymoops.txt [17:27] is nihals crash [17:27] okay Nihal is still there I asume? [17:27] yes [17:27] hi :) [17:27] hi! [17:27] its driving me crazy, the machine is crashing once every day! [17:28] he was silent so i figured that i would bring the subject up [17:28] UP or SMP? [17:28] first i thought one vserver went wild so i set on all vservers ulimits [17:28] UP [17:28] JonB thx :) [17:28] Nihal: you're welcome [17:29] well i have already found out that the lvm tools where faulty [17:29] or better [17:29] oh my god, that is really something ... [17:29] to old [17:29] i dont know why but lvm tools where 1.0.rc1 [17:29] it looks like you are suffering constant slab corruption ... [17:30] until now, I only know 2 causes for this ... [17:30] xsbyme (xsbyme@D5E0EDEB.kabel.telenet.be) joined #vserver. [17:30] now i checked all lvm partitions (they had all errors) and im now using lvm1.0.7 [17:30] k [17:30] hi xsbyme! [17:30] which ones? [17:30] a) your memory (cache) is faulty [17:30] hi Bertl [17:30] b) your compiler is buggy [17:30] hm [17:30] k [17:30] okay which compiler version? [17:30] well i am running gentoo [17:31] gcc -v will tell you ... [17:31] hm, one sec, have to check that [17:31] gcc version 3.2.3 20030422 (Gentoo Linux 1.4 3.2.3-r3, propolice) [17:32] okay, 'downgrade' to 2.95 or something like this or up to at least 3.3.2 ;) [17:32] heh, ok [17:32] for kernel compiling, I would suggest the former ... [17:32] well lets see if gentoo can do this for me :] [17:32] you can have 2.95 and 3.2.3 side by side ... [17:32] can i set which it uses by default? [17:32] yes ... [17:33] Nihal: you can tell in the Makefile which gcc to use [17:33] Nihal: have you tested memory with memtest86 yet? [17:34] Bertl in the beginning we tested it once.. it was ok [17:34] we then sent it to the housing center [17:34] but there we had to replace the motherboard because it suddenly was faulty [17:34] Nihal: and they can have trashed it by static [17:34] :( [17:34] @nihal hmm, okay how long did you test? [17:34] yes [17:34] well [17:35] another issue is that i have a irq routing conflict with the ide controller (i just discovered that) could that be a problem too? [17:35] hm, well about 5 hours i think [17:35] hmm, please elaborate ... [17:36] so Arvind-NL, is there too, what do you need? [17:36] elaborate? what do you mean? :) [17:36] the ide controller routing issue ;)? [17:37] ah ok, yes ill do that, but i have currently no idea how to fix it [17:37] PCI: Found IRQ 3 for device 00:0f.1 [17:37] IRQ routing conflict for 00:0f.0, have irq 12, want irq 3 [17:37] oops ... [17:37] using ACPI ? [17:37] _fleshcrawler (~fleshcraw@port-212-202-16-246.reverse.qsc.de) joined #vserver. [17:37] fleshcrawler (~fleshcraw@port-212-202-16-246.reverse.qsc.de) left irc: Read error: Connection reset by peer [17:37] actually no.. disabled acpi as far as i know [17:38] Nick change: _fleshcrawler -> fleshcrawler [17:38] what options I need to enable in the kernel config after applying patch-2.4.22-3-vs1.21.diff.gz [17:38] :-) [17:38] What options do I need to enable in the kernel config after applying patch-2.4.22-3-vs1.21.diff.gz . [17:38] Arvind-NL: simple you might need vroot device if you do quota/disk limits, but nothing else [17:39] Nihal: you should enable local APIC and UP APIC ... [17:39] in ther kernel config and ACPI as well ... CPU + termal ... [17:40] well i had on other machines issues with acpi so i disabled it.. is it really safe to enable it?= [17:40] hmm, okay what is your 'management' access to that machine? [17:40] ssh :> [17:40] you ahve somebody in the datacenter? [17:40] not today [17:41] tomorow [17:41] you have no serial line, no remote reset? [17:41] yes i have a remote reset [17:41] through apc [17:41] okay that should be sufficient ... [17:41] Bertl: he's hosting at ABSOLUTELYNOSERVICEATALL.com [17:41] hey cool! [17:41] lol [17:41] ? [17:42] apc ? [17:42] um, well i can reset the machine through the UPS .. its called apc or so.. ;) [17:42] lilo or grub? [17:42] lilo [17:43] okay, you compile a new kernel (no vserver code yet) [17:43] is it possible to do that, and have it run for some time? [17:43] Nihal: but you can not tell it which kernel to boot, right ? [17:43] hm, not really, our costumers would kill me ;) [17:43] @JonB that isn't a problem ... [17:43] JonB no i cant .. [17:43] yes, you can, with the -R option ;) [17:44] oh [17:44] the -R option ? [17:44] u mean tell lilo to boot once the new kernel [17:44] and if it doesnt work the old kernel? [17:44] /sbin/lilo -R - set default command line for next reboot [17:44] live and learn! [17:44] coool, i'm learning something new every day [17:44] yay [17:44] :) [17:45] @JonB vserver irc channel pays off, doesn't it? [17:45] Bertl: it sure does, thats why i keep comming back every day [17:46] s i activated the local apic and the io apic .. but what do you mean with cpu+termal? [17:46] @Nihal okay so testing isn't a good choice for now ... [17:46] Bertl: when you're arround bright intelligent people, you risk learning something [17:46] Bertl nope :> [17:47] good, then just add the apic stuff, and enable the nmi_watchdog in lilo (with nmi_watchdog=1 or nmi_watchdog=2) [17:47] and dont activate acpi? [17:48] leave it off for now ... [17:48] k [17:48] hm, where do i put the nmi_watchdog ? [17:48] just anywhere in the lilo.conf? [17:49] and do a fresh unpack/build when you changed the compiler ... [17:49] you add it like root= [17:49] ah ok [17:49] bad example for lilo, you put it in the append line ;) [17:50] what does this do? [17:50] so append=".... nmi_watchdog=1" [17:50] yes [17:50] this enables a software watchdog triggered by the non maskable interrupt (nmi) [17:51] if your system locks up, this causes a panic and if you add panic=10 (for example) a reboot after 10 secs ... [17:51] Bertl: does it actualy work ? [17:51] yup, it works, if the nmi_watchdog can be enabled ... [17:51] ok, so i set nmi_watchdog=1 and panic=5 or whatsoever in the append line :) [17:52] exactly ... we'll see after the first reboot, if nmi_watchdog=2 would be required ... [17:52] and it might be possible that both fail, but we'll find out after the second reboot ... [17:53] Bertl: why would they fail ? [17:53] on UP systems, there is a special APIC feature required and some performance counters to do that, not all systems have both ... [17:53] IRQ routing conflict for 00:0f.0, have irq 12, want irq 3 [17:53] with devices is that [17:55] @xsbyme hmm yes? [17:55] xsbyme thats my ide controller... [17:56] oh damn.. [17:57] hm, Bertl thanks for your help, but it looks like i have another emergency. [17:57] :( [17:57] okay, cu later ... [17:57] i think he asks, which, and not with [17:57] are u sometimes around here tomorrow? :) [17:57] might be, just look for the nick, Bertl_zZ is sleeping Bertl_oO unavail ... [17:57] hehe [17:57] ok [17:57] otherwise send me an email ... [17:58] ah, whats your email? :) [17:58] just in case ;) [17:58] herbert@13thfloor.at [17:58] ok, thx alot for your help! [17:58] cya [17:58] UR welcome! [18:00] xsbyme: what dou you need? [18:01] just idling [18:01] okay, no troubles with vserver then, good ... [18:03] Arvind-NL: what is your status? [18:03] do you need anything? [18:15] okay, seems like that's it for the moment ;) [18:19] Arvind-NL (~Hoi@ip503d4396.speed.planet.nl) left irc: Remote host closed the connection [18:22] heh [18:39] Tamama (~bgbgbg@a62-216-20-152.adsl.cistron.nl) left #vserver. [18:43] Nihal (force@brln-d9ba1fed.pool.mediaWays.net) left irc: Ping timeout: 485 seconds [18:46]