[00:00] Task had reference!!!! f6ccc000 2 [00:00] Task had reference!!!! f680c000 2 [00:00] tons of them [00:00] okay, so this is possible, and we might race ... [00:00] now the $1mio question, why can't you crash it? [00:00] im not sure :( [00:01] i did before [00:01] okay, which killer(s) did you try? [00:01] 75 processes: 33 sleeping, 42 running, 0 zombie, 0 stopped [00:01] CPU0 states: 9.1% user 89.0% system 0.0% nice 0.0% iowait 0.1% idle [00:01] CPU1 states: 7.0% user 91.0% system 0.0% nice 0.0% iowait 0.1% idle [00:01] CPU2 states: 9.1% user 90.0% system 0.0% nice 0.0% iowait 0.0% idle [00:01] CPU3 states: 7.0% user 92.0% system 0.0% nice 0.0% iowait 0.0% idle [00:01] killer-nathan without ip [00:01] let me try the original [00:03] nada [00:03] hmm, you ahve a proc loop now? [00:03] 30 of them [00:03] heh [00:03] in ctx 1 [00:04] hmm, okay, lets adjust the probabilities ... [00:06] add a loop right after [00:06] if (task->s_info) { [00:06] int i; [00:06] which tries 10000 times to verify task->s_info [00:06] k [00:06] and if it is still the same, continues without any comment [00:07] don't make it fast, make it complicated ;) [00:14] are you working on stable Bertl? [00:14] Action: johnny hopes not [00:14] yes [00:15] hmm, why do you hope, I'm _not_ working on it? [00:15] for (i=0; i<10000; i++) { [00:15] if (!task->s_info) { [00:15] printk("Vanished from under us here we go...\n"); [00:15] break; [00:15] } [00:15] } [00:15] ehrm, not quite ... [00:15] nope? [00:15] we asume that task->s_info is set .. at the beginning, right? [00:15] yes [00:16] so we wait for it to vanish from under us? [00:16] but we can't assume that it becomes zero later [00:16] could become anything ... [00:16] oh ok [00:16] we should check against a copy ... [00:16] yep [00:16] of course, it should become zero in many cases ... [00:17] cuz if it's broken... i will be scared.. [00:17] well johnny, nothing is perfect (at least almost nothing) [00:17] hm [00:17] for (i=0; i<10000; i++) { [00:17] cur_sinfo=task->s_info; [00:17] if (cur_sinfo!=orig_sinfo) { [00:17] printk("Vanished from under us here we go...\n"); [00:17] break; [00:17] } [00:17] } [00:17] that more on target? [00:18] unless you want me to do a memcmp to make things real slow :) [00:18] looks good ... [00:18] you could add a %p showing the new s_info [00:18] into that printk [00:19] printk("Vanished from under us here we go %p %p...\n", cur_sinfo, orig_sinfo); [00:19] and might change the break into a return ... [00:19] with the input buffer value ... [00:19] ah i was just gonna let the oops happen [00:19] okay [00:19] but i can get out clean i guess [00:19] nope, it's okay ... [00:19] k [00:28] got all kinds of oops [00:28] damnit shoulda teed it [00:28] i dont see my message though [00:33] >>EIP; c017f2f9 <===== [00:33] no stack [00:39] _MedivhWrk (ck@netops.multimedia-centrum.de) joined #vserver. [00:39] _Zoiah (Zoiah@matryoshka.zoiah.net) joined #vserver. [00:40] WSU (~Josh@ny.webpipe.net) got netsplit. [00:40] MedivhWrk (ck@netops.multimedia-centrum.de) got netsplit. [00:40] Zoiah (Zoiah@matryoshka.zoiah.net) got netsplit. [00:40] serving (~serving@213.186.191.116) got netsplit. [00:40] Nick change: _MedivhWrk -> MedivhWrk [00:40] Possible future nick collision: MedivhWrk [00:43] WSU (~Josh@ny.webpipe.net) returned to #vserver. [00:43] serving (~serving@213.186.191.116) returned to #vserver. [00:49] Bertl: Ayt? [00:51] Zoiah (Zoiah@matryoshka.zoiah.net) got lost in the net-split. [00:56] yup! [01:00] Do you usually run qemu from root or from some user? [01:00] usually root ... [01:00] but I guess it should not make a big difference ... [01:01] No. I'm thinking of more correct permissions for /dev/net/tun [01:02] well, it's a little problematic, but a group 'tunuser' should do the trick ... [01:02] You're probably right. [01:03] >>EIP; c017f28e <===== [01:03] >>ecx; f68e0000 <_end+36420bfc/38377c5c> [01:03] >>edx; c037d09a [01:03] >>esi; f6bf8165 <_end+36738d61/38377c5c> [01:03] >>edi; f6bf8153 <_end+36738d4f/38377c5c> [01:03] >>ebp; f6b5bf60 <_end+3669cb5c/38377c5c> [01:03] >>esp; f6b5bef8 <_end+3669caf4/38377c5c> [01:03] Trace; c017c830 [01:03] Trace; c014dfb6 [01:03] Trace; c0158968 [01:03] Trace; c010995f [01:04] a chance for a addr2line on proc_pid_status+3ae/5e0 [01:04] /usr/src/bertbrutal/linux-2.4.24/fs/proc/array.c:315 [01:04] which reads? [01:04] 314 for (i=0; i 315 int ctx = task->s_info->vx_id[i]; [01:04] 316 [01:05] hmm, but not your loop? [01:05] loop never dereferences s_info so it must not be changed [01:05] im not sure [01:05] well, there is a check a few lines above that if ->s_info is != 0 [01:06] or at least should be ... [01:06] 303 if ((orig_sinfo=task->s_info)) { [01:06] 304 int i; [01:06] 305 for (i=0; i<10000; i++) { [01:06] 306 cur_sinfo=task->s_info; [01:06] 307 if (cur_sinfo!=orig_sinfo) { [01:06] 308 printk("Vanished from under us here we go %p %p...\n", cur_sinfo, orig_sinfo); [01:06] 309 break; [01:06] 310 } [01:06] 311 } [01:06] what's in 290-302 [01:07] 290 buffer = task_name(task, buffer); [01:07] 291 buffer = task_state(task, buffer); [01:07] 292 task_lock(task); [01:07] 293 mm = task->mm; [01:07] 294 if(mm) [01:07] 295 atomic_inc(&mm->mm_users); [01:07] 296 task_unlock(task); [01:07] 297 if (mm) { [01:07] 298 buffer = task_mem(mm, buffer); [01:07] 299 mmput(mm); [01:07] 300 } [01:07] 301 buffer = task_sig(task, buffer); [01:07] 302 buffer = task_cap(task, buffer); [01:07] ahh okay, you put it before? [01:07] Action: nathan_ scratches head [01:07] before what? [01:07] hmm ... wait let me look at the original code ... [01:07] if (task->s_info) { [01:07] no, after [01:07] int i; [01:07] 303 if ((orig_sinfo=task->s_info)) { [01:08] but i changed it to that to get the original value [01:08] okay ... [01:09] ah, okay, maybe that gives some hints too ... [01:10] maybe sprintf() is a little tired sometimes ... [01:10] netrose (john877@CC3-24.171.21.47.charter-stl.com) joined #vserver. [01:11] is that reasoning for why my code never printed anything? [01:12] hmm, well, no it doesn't look like it can sleep, maybe it's only so much slower ... [01:12] box never went down hard though [01:12] i rebooted it [01:12] try to replace your loops by some simpler checks ... [01:12] it sync'd to disk and many oops made it, along with other printks from the refcounting [01:13] after each sprintf() compare the task->s_info with the orig_sinfo [01:13] strangely enough strings vmlinux is not showing 'Vanished' [01:14] its not being optimized out or something is it? [01:14] could be .. upload the entire code somewhere please ... [01:14] (this file) [01:14] http://0x00.org/hidden/array.c [01:16] what compiler version? [01:17] 3.2.2 [01:17] ill try 2.9.6 [01:17] hmm, please do so ... [01:17] and change that to some descrete checks after each sprintf [01:18] like what? [01:29] hmm, my connection was gone ... back again [01:29] hmm 2.9.6 has the Vanished string [01:29] what discrete checks did you want me to add? [01:29] yes, it might be that the 3.2.2 optimized i away, because task wasn't define as being volatile ... [01:30] ah [01:30] and dereferencing the same element, is assumed to give the same results ... [01:30] yea [01:30] okay, what I meant was doing some 'simple' [01:30] if (orig_sinfo != task->s_info) [01:31] printk("ouch .. to bad %p != %p\n" [01:31] well you get the idea ... [01:36] hmm well i ran it with 2.9.6 but i dindt change any of the code, not getting an oops [01:36] but i had trouble reproducing the oops again on that last kernel the second time also [01:36] I almost supected that ... [01:36] maybe we are hunting compiler issues here ... [01:37] hmm [01:37] I dont trust 3.2.x it broke to many kernels for me ... [01:37] you know, iirc my original tests did not use killer [01:37] let me try my original tests [01:37] make that ... [01:52] what's the split bast ? [01:52] patch* [01:52] which one? [01:53] there's split and a regular patch [01:53] for 2.4.23 [01:53] ahh, okay ... [01:53] the split-* is a tar of some patches, together producing the patch-* patch [01:53] is there a problem in applying grsec first ? [01:53] this is to ease development and/or modifications ... [01:54] monako (~monako@ts1-a20.Perm.dial.rol.ru) joined #vserver. [01:54] well, if it works, there is no problem ... [01:56] netrose (john877@CC3-24.171.21.47.charter-stl.com) left #vserver. [01:56] netrose (john877@CC3-24.171.21.47.charter-stl.com) joined #vserver. [02:02] Bertl: i've googled a around a little and it seems like there is a problem with local masquerading... [02:02] no there is no problem with local masquerading, it just doesn't exist ... [02:03] okay [02:03] but how can i get it to work then? no way? [02:03] there was a patch floating around, which processed local originating packets with the 'normal' netfilter code .. IIRC [02:04] and you can always use SNAT for most purposes ... [02:04] /quit zzzzzzzzzzz [02:04] JonB (~jonbendts@129.142.112.33.ip.tele2adsl.dk) left #vserver. [02:05] nathan_: I'm pretty sure you are right, and proc_pid_status() is racy ... [02:06] okay i'll go find out about SNAT ;) [02:06] spunk: read the netfilter howto ... [02:06] and Advanced Linux networking ... [02:07] okay i'll do that [02:07] for vserver, assume every trafic is generated locally' if you use a local network on your host ;) [02:08] iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -j SNAT --to-source MY_OUTSIDE_IP [02:09] something like this? [02:09] Bertl, yea, just having trouble reproducin it on demand atm :/ [02:09] yes, but be careful, they will get the same ip, regardless of the ports ... [02:11] i guess i have to read the howtos to understand properly what im doing ;) [02:20] nathan_: maybe putting some schedule() just between the critical proc reagons could trigger it? [02:21] hmm worth a shot [02:47] got an oops [02:47] and another [02:48] good, okay ... [02:48] Vanished from under us here we go 00000000 f6ccf200... [02:48] however, i did schedule() inside of the proc_status [02:49] not sure if it creates a side affect? [02:49] well, yes, it might change the cpu ;) [02:49] so, maybe not such a great idea where i placed it? [02:50] but the vs1.2.2.2 should avoid that [02:50] (that race, not the schedule) [02:50] now the trick part will be to show that vs1.2.2.2 still has issues ... [02:50] 1.2.2.2 is what we easily crashed isnt it? [02:51] well, try again with gcc 2.95/96 [02:51] maybe this time we get a better stack backtrace ... [02:52] or do you ahve a better idea? I don't see the race in vs1.2.2.2 .. an those races seem fixed there ... [02:53] i dont see the problem either :/ [02:53] ill try ith 296 [03:38] monako (~monako@ts1-a20.Perm.dial.rol.ru) left #vserver (всем пока ... good bye all ...). [03:43] no workie Bertl ... :( [03:44] gonna have to use 2.4.23 and manually patch grsec in.. [03:44] hmm .. there are grsec patches for 2.4.23 on linux-vserver.org [03:44] yep [03:45] why shouldn't they work for 2.4.24? [03:45] i meant with 2.4.23 [03:45] Bertl, would you be angry if i told you that this doesnt seem to be crashing with 296? :) [03:45] i did emerge grsec-sources [03:45] so vserver didn't apply on top of it [03:45] looks like vserver has to be first if i wanna avoid rejects [03:46] guess i'll see [03:46] ill build a clean tree with 2.9.6 and 3.0 and see if i can reproduce it on demand [03:46] nathan_: no, I would be happy [03:46] Action: Bertl is kicking agains the waste bucket ... [03:47] nathan_: no, not a problem with that .. if it works and is stable, we just learned something ... [03:48] the grsec + vserver patch is only for 2.4.23 anyways [03:48] as seen here http://www.firehead.org/~jeffrey/linux-vserver/ [03:48] johnny: is emerging the level of your kernel patching skills? [03:48] (no offense meant by that) [03:48] nope.. although i'm no expert because emerge made my life so easier [03:49] i patched many a 2.5.x kernel [03:49] johnny, grsec is not a hard patch into 2.4.23/24 with vserver [03:49] okay, what _are_ the actual rejects on 2.4.24? [03:49] its signal handling that needs to be changed [03:49] i didn't mess with 2.4.24 [03:49] the rest is minor [03:49] some proc stuff [03:49] mostly signal handling is the only stuff that needs to be reviewed [03:49] because of no grsec + vserver patch here http://www.firehead.org/~jeffrey/linux-vserver/ [03:49] for 2.4.24 [03:50] i don't like mixing up kernel versions if possible [03:50] should that patch apply on a 2.4.24 ?\ [03:50] try it [03:50] most likely it will and only minor rej [03:50] maybe just Makefile if you are lucky [03:51] hurmpf [03:51] well first i need to patch it for vserver .. [03:51] hmm [03:51] i did want the aavm though ...:( [03:52] guess i'll wait for kerfy to come back around [03:52] Bertl, i got a curious nmi_watchdog oops after i think issuing a reboot [03:52] he's made a pretty nice patchset off of -uw [03:52] but im gonna try a 3.0 version now [03:52] that ultra stable patchset .. + aavm [03:59] hmm [03:59] not sure why the vserver patche failed [04:00] it looks like it would have succeeded [04:00] on the makefile [04:00] - EXTRAVERSION = + EXTRAVERSION = -vs1.22 [04:00] why the heck would that fail.. [04:01] not that it matters much [04:05] hmm... wonder why the listing says 5 rej... [04:05] and when i do find . -iname *.rej i only get 2 .. [04:11] ahh.. [04:11] hmm [04:23] spunk (~spunk@c173134.adsl.hansenet.de) left irc: Quit: My damn controlling terminal disappeared! [06:04] hey bert [06:04] yup? [06:05] do we have issues with sysctl interface in addition to the proc? [06:05] might be .. [06:05] do you mean access or race? [06:05] access [06:06] hmm, that is controlled by some caps IIRC [06:06] so just for the record, the final state (for today) is: [06:06] vs1.2.2.2 is stable, right? [06:07] seems to be when compiled with 296 [06:07] would be nice if Tamama could confirm [06:07] okay, keep compiling with 2.96 [06:09] maybe he does later .. (I hope so) [06:27] bert have you done any further work on the proc security stuff? [06:27] yup, it's almost done .. [06:27] AH [06:27] err ah [06:45] okay, I go to bed now, guess we'll do a 1.23 and 1.3.5 tomorrow ... [06:46] do you have anything i can take a look at? [06:46] anything Experimental yet? [06:46] hmm, still want to work? [06:46] yea its only 10:48 here, i want to play with the new proc stuff [06:47] okay ... [06:50] http://vserver.13thfloor.at/Experimental/patch-2.4.24-vs1.3.4.3.diff [06:50] http://vserver.13thfloor.at/Experimental/vxid-0.01.tar.bz2 [06:50] okie [06:50] I'm confident, you figure out what to do, yes? [06:50] ill play [06:50] yep [06:50] have a good night [06:50] oki, have a good one too, cu ... [06:50] and thanks for all your help! [06:51] Nick change: Bertl -> Bertl_zZ [07:18] Doener (~doener@pD9588CB6.dip.t-dialin.net) got netsplit. [07:18] _Zoiah (Zoiah@matryoshka.zoiah.net) got netsplit. [07:18] kestrel (athomas@dialup51.optus.net.au) got netsplit. [07:18] riel (~riel@riel.netop.oftc.net) got netsplit. [07:18] Bertl_zZ (~herbert@MAIL.13thfloor.at) got netsplit. [07:18] sladen (paul@starsky.19inch.net) got netsplit. [07:18] ccooke (~ccooke@80.1.164.238) got netsplit. [07:18] nathan_ (~nathan@209-6-130-26.c3-0.sbo-ubr1.sbo-ubr.ma.cable.rcn.com) got netsplit. [07:18] kungfuftr (~scott@jadis.narnia.org.uk) got netsplit. [07:18] WSU (~Josh@ny.webpipe.net) got netsplit. [07:18] serving (~serving@213.186.191.116) got netsplit. [07:18] _shuri (~shushushu@vserver.electronicbox.net) got netsplit. [07:18] netrose (john877@CC3-24.171.21.47.charter-stl.com) got netsplit. [07:18] netrose (john877@CC3-24.171.21.47.charter-stl.com) returned to #vserver. [07:18] serving (~serving@213.186.191.116) returned to #vserver. [07:18] WSU (~Josh@ny.webpipe.net) returned to #vserver. [07:18] _Zoiah (Zoiah@matryoshka.zoiah.net) returned to #vserver. [07:18] kestrel (athomas@dialup51.optus.net.au) returned to #vserver. [07:18] _shuri (~shushushu@vserver.electronicbox.net) returned to #vserver. [07:18] Doener (~doener@pD9588CB6.dip.t-dialin.net) returned to #vserver. [07:18] riel (~riel@riel.netop.oftc.net) joined #vserver. [07:18] Bertl_zZ (~herbert@MAIL.13thfloor.at) returned to #vserver. [07:18] sladen (paul@starsky.19inch.net) returned to #vserver. [07:18] ccooke (~ccooke@80.1.164.238) returned to #vserver. [07:18] nathan_ (~nathan@209-6-130-26.c3-0.sbo-ubr1.sbo-ubr.ma.cable.rcn.com) returned to #vserver. [07:18] kungfuftr (~scott@jadis.narnia.org.uk) returned to #vserver. [11:37] loger joined #vserver. [11:42] MrBawb (abob@swordfish.drown.org) joined #vserver. [11:43] johnny (~johnny@ip68-10-185-29.hr.hr.cox.net) joined #vserver. [11:43] CosmicRay (~jgoerzen@glockenspiel.complete.org) joined #vserver. [11:53] Doener_zZz (~doener@pD95889E8.dip.t-dialin.net) joined #vserver. [12:00] serving (~serving@213.186.191.116) left irc: Ping timeout: 480 seconds [12:01] Doener (~doener@pD9588CB6.dip.t-dialin.net) left irc: Ping timeout: 492 seconds [12:02] nathan_ (~nathan@209-6-130-26.c3-0.sbo-ubr1.sbo-ubr.ma.cable.rcn.com) left irc: Quit: BitchX-1.0c20cvs -- just do it. [13:29] deadguy (deadguy@bananajoe.big.du.se) joined #vserver. [13:53] serving (~serving@213.186.189.43) joined #vserver. [14:37] serving (~serving@213.186.189.43) left irc: Ping timeout: 504 seconds [15:23] johnny (~johnny@ip68-10-185-29.hr.hr.cox.net) got netsplit. [15:24] johnny (~johnny@ip68-10-185-29.hr.hr.cox.net) returned to #vserver. [15:41] Nick change: Bertl_zZ -> Bertl_oO [16:00] oO? [16:08] serving (~serving@213.186.189.43) joined #vserver. [16:14] JonB (~jonbendts@cpe.atm2-0-1041178.0x50a614da.bynxx14.customer.tele.dk) joined #vserver. [16:39] morning [16:51] i b0rked my kernel with vs+grsec :( [17:06] johnny: that was unlucky' [17:06] surely was [17:06] still trying to work out what was the cause... [17:38] johnny: vs+grsec? [17:38] works fine here [17:38] both in prod and on my testservers [17:50] hmm... [17:50] i'm gonna try again soon [17:50] Action: johnny hopes it wasn't the network card... [17:50] i wonder if my options are too restrictive.. [17:50] i doubt it though.. [17:52] what kind of setup do you run on grsec then? [17:57] <_Zoiah> I'm using vserver+openwall. [17:57] <_Zoiah> I'm not using gradm anyhow, and I they patch cleanly together. [17:57] openwall? [17:57] thats not grsec based now, is it? [17:57] <_Zoiah> No. [17:58] well, wonder how you mix that into this issue? :D [17:58] <_Zoiah> But it does give me proc restrictions, Non-executable user stack area, etc. [17:58] :) [17:58] well, grsec is a bit stricter in some stuff, both due to that theres a harder acl-system [17:58] <_Zoiah> The ACL system is incompatible with vserver, afaik? [17:59] works fine here [18:24] johnny (~johnny@ip68-10-185-29.hr.hr.cox.net) left irc: Remote host closed the connection [18:45] <_Zoiah> Ahh. [18:45] <_Zoiah> I heard there were incompatibilities with gradm and vserver. [18:45] <_Zoiah> Oh well, I don't really use that anyhow, because using it properly would give me way too much administrative overhead. [18:50] Finidi (~F-i-n-i-d@pD9003856.dip.t-dialin.net) joined #vserver. [18:51] Finidi (~F-i-n-i-d@pD9003856.dip.t-dialin.net) left #vserver. [18:55] serving (~serving@213.186.189.43) got netsplit. [19:06] serving (~serving@213.186.189.43) got lost in the net-split. [19:11] johnny (~johnny@ip68-10-185-29.hr.hr.cox.net) joined #vserver. [19:20] later [19:20] JonB (~jonbendts@cpe.atm2-0-1041178.0x50a614da.bynxx14.customer.tele.dk) left #vserver. [19:38] quietness [19:41] on a sunday morn [19:41] quietness strikes #vserver [19:41] all have a real life [19:42] oops, a syllable too much [19:45] hmm [19:45] wonder how i can set this thing up to fallback to the old kernel if the network card doesn't come up [20:53] serving (~serving@213.186.189.43) joined #vserver. [21:05] lilo -R ? [21:14] Nick change: Bertl_oO -> Bertl [21:14] hi everyone! [21:17] <_shuri> hi Bertl [21:18] hey shuri, how is your weekend? [21:18] <_shuri> fine [21:18] <_shuri> i am sick but is fine [21:18] <_shuri> :P [21:19] hmm, sick like cold, or sick like really sick? [21:19] <_shuri> realy sick [21:19] <_shuri> congestion [21:19] <_shuri> like hell [21:19] <_shuri> heheh [21:24] well, that no fun ... I guess ... [21:39] Bertl: Another question on qemu, can I? :) [21:40] yup, sure ... go ahead [21:41] I've just compiled 2.6.1-vs0.04 and qemu says: mmap failed when mapped physical address 0x00100000 to virtual address 0xc0100000 [21:42] After all, I noticed this on every kernel except those they suggest on qemu site. [21:42] hehe, you have to 'patch' the kernel for qemu .. [21:42] I may need some further rtfm though. [21:43] (unless you use the soft-mmu version which is slow) [21:43] Oh, thanks. [21:44] http://vserver.13thfloor.at/Stuff/QEMU/ [21:44] Hm. soft-mmu just hangs eating up 100% cpu. [21:45] Or maybe it uncompresses itself for too long? [21:45] I mean kernel. [21:47] nathan_ (~nathan@209-6-130-26.c3-0.sbo-ubr1.sbo-ubr.ma.cable.rcn.com) joined #vserver. [21:47] morning [21:48] nathan_: [21:51] hey nathan_ ! [21:51] heya [21:55] did you test the proc stuff? [21:56] after an accident with a gluegun last night i couldnt type so i never got it [21:57] but im healed and i will play with it today [21:58] hmm, shall I ask what kind of accident? ;) [21:59] sticking my hand on to a line of hot glue [21:59] mmmm feels good [22:11] Bertl, i got my kernel patched with grsec + vs ... now i wanna try one more time to get it to work..do you know how i could setup something in the init that will do a sleep for like 30 seconds... if i don't login and kill it.. it will restart the computer with the old version of my grub.conf ? [22:11] and i don't want it stopping init either.. that would be pointless [22:12] wonder if a simple sleep.sh & will work... [22:12] i'ts on a dedicated server... so i can't access it directly [22:12] ( sleep 1200; shutdown -h now ) & [22:13] but try it first with a working kernel ... [22:13] and use panic=10 on the kernel command line, if it panics, it will reboot after 10 secs ... [22:14] johnny, does grub have anything like lilo -R? [22:15] yes, it has ... [22:15] it didn't panic [22:15] it came up [22:15] but won't that make it retry the same kernel again? [22:15] bert will tip you off to the lilo -R method of grub and then it will be a boot-once [22:15] i dont use grub so i dont know it [22:16] johnny, rackshack? [22:16] no.. [22:16] ah [22:16] just curious [22:16] united colo [22:16] or sagonet.. [22:16] echo "savedefault --default=$number --once" | grub --no-curses --device-map=/boot/grub/device.map [22:16] or you can enter the grub shell and do [22:17] savedefault --default= --once [22:17] some 'older' versions of grub seem to fallback to default=0 after that ... [22:23] ok.. it worked... [22:24] hmm, great! [22:26]