From: Charles Dale (bug_at_aphid.net)
Date: Fri 31 Oct 2003 - 14:30:55 GMT
Sure! I'll send these instructions in an email first, then put them up on
the web if they prove useful. Hopefully that way they'll improve with time
that way. Note that it was generally due to quirks in the original config
that I had to fix things in the vserver. It was very easy in general to get
working. Yay vserver!
I'm assuming that you want to do what I've done - run an older distro inside
a newer one. There's nothing special about doing this as opposed running the
same distros on the host server & vserver, it's just the idea's a bit
weird... I assume it's possible that the newer kernel & older glibc might
clash, but I haven't had any problems yet, and the RH6.2 box was running a
2.2.18 kernel, whereas now it's under 2.4.22.
(I used the c17f patch with Jack's vserver-0.25. Yes I'm going to switch to
Modifications needed (people please correct me anyone if there were better
ways to do these things):
On the host server (RH9):
* Make 127.0.0.1 available inside the vserver. As I had many config files
explictly binding to 127.0.0.1 I thought it would be easier to make that
available in the vserver. I did this by changing the lo interface in the
host vserver to have IP 127.0.0.2, then adding lo:127.0.0.1 to the IPROOT
config setting. Seemed to work alright. I've seen no docs explaining what to
do about localhost in the vservers... (anyone care to explain?)
* Add CAP_SYS_RESOURCE to S_CAPS to allow BIND to work in the vserver.
* Remove "nproc" from S_FLAGS in the vserver config & set ULIMIT="".
Originally ulimit wasn't working (I think there's another kernel patch that
I need to apply), so I set ULIMIT="". But then I found the vserver seemed to
have a very low process limit for some reason. So I removed "nproc" and
things have been fine since then. I'm sure that wasn't really the right
thing to do, but it was 4am and the clock was ticking. (Anyone have any
hints on what I should do to get ulimit set correctly?)
In the vserver (RH6.2):
* Edit /etc/rc.d/init.d/syslog, commenting out the lines that start and stop
klogd (only the host server should get the kernel log messages).
* Turn off useless (in a vserver) services: network, rawdevices, random,
ntp. (to do this run, for example, "chkconfig --del network")
* Remove various kill scripts from /etc/rc.d/rc3.d/ that were hanging
around. Kill scripts start with a K. These were executing when I stopped the
vserver, trying to do things like disable the network interfaces. Not sure
why chkconfig --del didn't remove these.
* Fix the permissions on a few directories that were set to 0 (i.e. chmod 0
dir), because vserver uses chmod 0 to stop people breaking chroot. It took
me a while to work out why even as root I couldn't read anything in these
directories. A chmod u+rx fixed that.
* Remove /etc/rc.d/rc.sysinit. This kept trying to do things like add swap
space when I started the vserver.
* Edit /etc/rc.d/rc.local and remove any unnecessary lines (e.g. for me,
* Edit /etc/resolv.conf and change the nameserver setting to point to the
vserver's IP instead of 127.0.0.1. Not sure why this was needed, but name
resolution was very slow until I changed this. Maybe I shouldn't have played
with the loopback address...
* I also had problems at the start where inet & httpd refused to bind to
their ports, saying someone else had those open. I think I must have still
had a test vserver running with an overlapping IP. Killing all processes in
the other vservers fixed this...
* "find / -user 25 | xargs chown named; find / -group 25 | xargs chgrp
named;" -- Needed this because rsync had decided to change the owner of
named's files to UID 25 (which is named's UID on the host server), from UID
11 (which was named's UID on the old server).
* "chgrp utmp /var/log/wtmp" -- as with named the rsync had changed the UID.
(A quick check of man rsync tells me I should have done the rsync with the
-p option - would have avoided this problem and the previous one.)
* Totally unrelated to vserver issue: sysstat decided to stop working
because I'd moved from a single CPU system to an SMP system. Fix by "rm
Finally, once everything was working well:
* "arping -S NEW.IP -c 5 BROADCAST.ADDRESS" to tell everyone the new MAC
address of the server.
THINGS STILL TO FIX:
* Need to install the per context quota patches & get that running so the
vserver won't be able to full up the disks of the main server.
* Need to work out what to do about quotas inside the vserver. I don't
_think_ per context quotas give me what I want here. I want to be able to
use quotas _inside_ the vserver just like I was using before, when /usr on
the rh6.2 box was a separate partition.
* Sort out the ulimit business as discussed above.
So all in all there aren't many general settings to be changed after you
rsync the server into a vserver. I found the resolver issues and the named
permissions problem took the most time to fix. Give it a go - there's not
much to lose!
[mailto:vserver-admin_at_list.linux-vserver.org] On Behalf Of Charles
Sent: Friday, 31 October 2003 3:48 PM
Subject: Re: [Vserver] 1.0 release - I agree!
can you tell me how to make it work?
do i need to modify the setting of the rh6.2 after "rsync"?
i've got a rh7.3/rh9 host server, and like to run a rh7.3/rh9 vserver
thanks a lot!
----- Original Message -----
From: "Chuck Dale" <chuck_at_yourweb.com.au>
Sent: Thursday, October 30, 2003 10:01 PM
Subject: [Vserver] 1.0 release - I agree!
> [Hi new list!]
> Just a confirmation that I've been running the c17f patch here without
> problems for a few weeks now. Xeon CPU, 2.4.22 kernel, RH9.
> I'm so happy with vserver, thanks everyone! Just did a great trick:
> our old RH6.2 server that was running on its lonesome on crusty
> a brand spanking new box. The RH6.2 install is running as a vserver
> inside the RH9 host server. It's so cool...
> I'd be happy to help out with docs especially now that I've had a bit
> of experience playing with the super vservers.
> Vserver mailing list
Vserver mailing list
Vserver mailing list