Re: [Vserver] Problem with high loadavg on vservers without real load

From: Herbert Poetzl <herbert_at_13thfloor.at>
Date: Thu 30 Nov 2006 - 18:52:01 GMT
Message-ID: <20061130185201.GM2826@MAIL.13thfloor.at>

On Wed, Nov 29, 2006 at 01:45:24PM +0100, Walter Brunner wrote:
> Hello,
>
> I am using vserver approx. 2 years for hosting almost all my services
> (web, mail, application services)
> A half year ago I changed my main servers to AMD Athlon 64 Processors
> (3700+). With the host operating system I stayed at 32-bit. I always
> used lvm2 with reiserfs and with the change I also switched to drbd to
> mirror the lvm-partitions to a slave servers for the important
> vservers running.
>
> Since the switch to this configuration the stability is not optimal.
> Troubles occur in average 1 time a month. Which is not OK, but I
> accepted it. But yesterday and the day before yesterday during night
> there were troubles on 2 machines within 10 minutes.
> What happens? The procedure is allways the same. I monitor the loadavg
> of all vservers. Here is a table of the loadavg for the host and 1
> guest.
>
> seconds loadavg_guest loadavg_host
> 0 0.8 0.25
> 30 3.3 10.77
> 60 7.5 20.6
> 90 11.8 30.4
> 120 15.7 39
> 150 16.6 41
> ... (increase about 2 in 30 seconds)
>
> The other guest on the machines are not affected and worked well. The
> affected guest had high timeouts with http-connections.
> I could not enter the affected guest with vserver ... enter. vserver
> ... stop did not work. vkill did also not work. (I always had to
> reboot per hardware-reset.)
>
> The troubles were always during night hours (with 2 exceptions) where
> the load on the servers is minimal.
>
> Here are the program versions I use
> Linux version 2.6.17.13-vs2.0.2.1 (gcc-Version 3.3.5 (Debian 1:3.3.5-13))
> Debian sarge on the host. Most guest are also sarge, some are gentoo,
> some are ubuntu 6.06
> drbd-0.7.21
> util-vserver-0.30.210
> kernel / drbd and util-vserver is self-compiled, all other packages
> (incl. lvm2) are from debian sarge main
>
> Does anyone have an idea, what's problem? I don't want to wake up
> every night per sms.

IIRC, that was somewhat dealth with yesterday on IRC,
but in general the following 'suggestions' can be made:

 - use the devel branch (2.1.x) or the upcoming stable
   branch (2.2.x) because certain things got quite
   improved there regarding scheduler and limits

 - make sure that your host can handle the resource
   consumption of the guests and that you have enough
   memory and a reasonably fast I/O path

 - make sure to limit all resources on potentially
   hostile guests, even network resources with tc

 - avoid script storms (e.g. by adjusting the cron
   tabs to not run at 4am for all guests at the same
   time)

 - remove unnecessary services for guests, and scripts
   which do not make sense (like hardware detection or
   sensor checks)

all-in-all, if the host system is designed to handle
the guests, the system will be stable and basically
free of any maintainance ... I know, the hard thing
is always to figure an optimal setup ...

HTH,
Herbert

> Walter
> _______________________________________________
> Vserver mailing list
> Vserver@list.linux-vserver.org
> http://list.linux-vserver.org/mailman/listinfo/vserver
_______________________________________________
Vserver mailing list
Vserver@list.linux-vserver.org
http://list.linux-vserver.org/mailman/listinfo/vserver
Received on Thu Nov 30 18:54:57 2006

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Thu 30 Nov 2006 - 18:55:10 GMT by hypermail 2.1.8