Re: [Vserver] Hosting Vserver's on CODA or Lustre Distributed File Systems?

From: Jon Scottorn <>
Date: Mon 14 Nov 2005 - 18:10:20 GMT
Message-Id: <1131991820.7076.3.camel@pf-dev>

We are running with a GFS setup currently and it works good for the most
part. Lustre will not work with vservers until the lustre designers get
their stuff working in the kernel. That is the route we prefer to go
but until that happens it just doesn't work. We have spent a great deal
of time with a clustered vserver setup and so far GFS is the only FS
that has worked with stability. The main problem with it is the locking
that it does and such so things get very bottle necked with the file
system. If you have further questions you can look at our previous
conversations dealing with clustered file systems.

Jon Scottorn
Systems Administrator
The Possibility Forge, Inc.
435.635.0591 x.1004

On Mon, 2005-11-14 at 08:42 -0800, EKC wrote:

> Hello,
> I have a group of nine Linux boxes each running several Vservers. Each
> Vserver is a self-contained webhost with Apache, MySQL, Qmail, etc...
> The Vserver "images" on each machine reside on the local hard disk.
> There are no network shares (AFS, etc...)
> Needless to say, it is becoming increasingly difficult to backup the
> Vservers. I am also concerned about redundancy and failover. Having
> the Vserver images all spread out is also an administrative pain. I am
> not using RAID and would like to use some sort of distributed
> data-store for the Vserver "images".
> I have looked at Redhat GFS, but ruled it out because of the high-cost
> of using a SAN and scalability problems. I would like to scale to
> several dozen machines running Vservers.
> The ideal solution would be a highly-scalable distributed file-system
> with built-in data redundancy (RAID-1 striping across the storage
> nodes). I would like to be able to shutdown a Vserver on one machine,
> and start it up on another without having to SCP the vserver directory
> over.
> So far, I have come up with only two solutions:
> The first is using CODA. I would set aside a group of 3-4 servers
> (including replicas) to host the CODA file-system. The Vserver images
> would be stored here. The "client" machines would mount the CODA
> file-system and run the Vservers. CODA includes a persistent local
> cache on each client to deal with latency issues. However, I'm worried
> that CODA may not scale well (based on discussions on the CODA mailing
> list).
> I am also looking at using the Lustre distributed file system
> ( Lustre seems to be actively supported, and is
> highly scalable (1000's of nodes). However, it does not yet support
> RAID-1 striping across storage nodes or persistent local cache's on
> the clients. So, I'm worried that latency may be an issue. Throughput,
> however, scales linearly on a Lustre cluster with the number of
> storage nodes (read: machines) -- a characteristic that CODA does not
> have. If Lustre is the best option, I could always go for software
> RAID on each storage node until Lustre supports RAID-1 striping.
> Has anyone run Vservers from a CODA or Lustre file system? If so, what
> is your experience? Does it scale well? Does CODA have scalability
> problems? Does Lustre have latency problems?
> Should I use CODA or Lustre?
> All advice/input is highly appreciated!
> Thanks in advance
> _______________________________________________
> Vserver mailing list

Vserver mailing list
Received on Mon Nov 14 18:11:12 2005

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Mon 14 Nov 2005 - 18:11:18 GMT by hypermail 2.1.8