2012/9/7 Gordan Bobic <gordan@bobich.net>:
> On 09/07/2012 04:11 PM, Tor Rune Skoglund wrote:
>> I been trying to read up on the hashify with COW feature of vserver.
>> Still some questions...:
>>
>> - What are the actual requirements for using this feature?
>> - Any particular file systems only?
>
> IIRC only ext* file systems are supported/patched (included in the vserver
> patch).
>
>> - Presumably, all hashifed files must reside on the same partition?
>
> Indeed, they must all be on the same file system.
Additional ?: The host run on a partition of it own. The guests on
another partition. So then the host is left out of the hashify
process, but the guests still can be hashified towards "eachother" ?
>> - What if the process of hashifying is interrupted? Will filesystems
>> still be left in good working order?
>
> Yes. The process hard-links all files to a file named by their hash under
> /vservers/.hash/. It then goes and replaces all the files one by one with a
> hard-link to this central hash location. The net result is that all the
> identical files will end up sharing the data blocks. There is nothing going
> on that could trash the file system.
OK, great.
>> - Can the hashify process be run on running vservers, or should they be
>> stopped before starting the hashify process?
>
> You can run it on the running server, but you will not gain the full benefit
> of memory deduplication until you re-start all the processes in all the
> guests (i.e. until all the file handles are re-opened).
OK. In our use-case, most systems will be restarted at least once a
workday, so that should then be covered.
>> lxc seem to prescribe btrfs as the lxc "equivalent"
>> (http://www.digipedia.pl/usenet/thread/18777/2045/ ). How does this
>> compare to vserver's hashify? (Some answer given in that discussion, but
>> still a couple of open ones...)
>
> I started that thread. :)
> It is _NOT_ equivalent. The key difference is that it won't give you memory
> deduplication for free. One of the big unique selling point of vserver is
> that by using CoW hard-links it gives you memory deduplication for free.
> Think about how mmap() works and how the memory address is picked - it is
> done by inode number. Hard-links have the same inode number. That means that
> mmap()-ing the same hashified file in different guests will all go to the
> same memory address. Free memory dedupe if all your guests run the same OS
> with the same package versions (Note: DISABLE AND UNINSTALL PRELINK!).
> Imagine having 100 guests and instead of needing 100 instances of glibc in
> RAM you only have one.
Yes, great.
> Normally memory deduplication is very expensive in terms of CPU time, but in
> vserver with hashify you get it for free.
>
>> - Can btrfs be used with vserver?
>
> Not unless somebody write the CoW hard-link patches recently.
OK, so if btrfs is selected, then the vserver solution with
hashify+CoW is out of reach.
>> - What about trebuchet (http://wiki.baserock.org/Trebucket ): Any
>> qualified guesses whether it will be compatible for updating a vserver
>> guest? (USE-CASE: Update a guest as an atomic operation; keep fallback
>> vserver guest ready in case update process incomplete, or in case new
>> vserver guest do not perform correctly.)
>
> That link seems to be dead, and I cannot figure out from paragraph context
> alone what it's supposed to achieve. Please elaborate?
Sorry, my explanation was incomplete. And the link lacks the /, so it
is actually http://wiki.baserock.org/Trebuchet/ .
Our use-case is a distributed system. We need to update a vserver
client as an atomic operation (and the host, by that is another
issue...)
More detailed, we would want to download a new "version" of the
vserver client as a delta that will "update" the current installation,
but keeping the possibility to revert to the pre-updated one in case
the update fails or if the vserver client is some way do not work
correctly or whatever reason. Trebuchet seems to offer that (probably
with some addition development and setup), BUT relies on btrfs.
So my initial point was; if we go down the Trebuchet road, then we
need to use btrfs, and then we will not be able to use vservers
hashify function (but still be able to have vserver without hashify).
Presumable my reasoning for that is correct?
Anyway, Gordan, thanks for great answers! :-)
- Tor Rune Skoglund
Received on Sat Sep 8 11:10:20 2012