Re: [vserver] "Stack smash" error running vserver under hardened AMD64?

From: Ed Wildgoose <lists_at_wildgooses.com>
Date: Fri 21 Mar 2008 - 23:39:19 GMT
Message-ID: <47E44727.8070509@wildgooses.com>

vitalyb wrote:
> Ed Wildgoose wrote:
>> Hi, can someone please give me some tips on vserver under hardened
>> amd64?
>>
>> I have a working 32 bit server running the latest 2.2 vserver+grsec
>> patch without any problems. I have taken the kernel .config and
>> switched it over to 64bit and recompiled on a new machine, then
>> rsynced over the guests to this new machine. When I try and start
>> them under the 64bit machine (they are 32 bit images obviously) I get
>> the error
>>
>> # vserver www2 start
>> stack smashed in main (value 0x00000000000kpp00)
>
> Had to track this down recently. I suppose it's Hardened Gentoo?
> This error happens because of dietlibc. "diet" wrapper do not
> passes "-fno-stack-protector" option to gcc (which has ssp
> enabled by default) and strips this parameter if you
> try to add it to CFLAGS manually.
>

Yep - hardened gentoo.

Some further investigation reveals that the ebuild already sets
-fno-stack-protector and -fno-stack-protector-all

The trick seems to be to change my CFLAGS and disable optimisation (ie
-O2). With CFLAGS including -O2 I get the stack smashed error above -
removing -O2 and keeping all other CFLAGS (-march=athlon64 -pipe) it
compiles and superficially at least seems to run ok...

So I'm guessing that gcc-3.4.6 optimisations are doing something which
triggers the problem rather than it being the code itself? Perhaps some
variable aliasing?

So just for reference (and benefit of google). You can compile up
util-vserver fine (not fully tested mind) if you compile dietlibc
without any -O flag (presumably defaults to -O1 ?)

Any other thoughts appreciated?

Ed W
Received on Fri Mar 21 23:39:37 2008

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Fri 21 Mar 2008 - 23:39:41 GMT by hypermail 2.1.8