Re: [Vserver] Multiple Routing Tables (Was: PPP inside VServer)

From: Eugene Roux <eroux_at_damn.org.za>
Date: Tue 05 Sep 2006 - 16:10:46 BST
Message-Id: <1157469046.4848.50.camel@norbert>

On Tue, 2006-09-05 at 08:46 -0500, Michael S. Zick wrote:
> On Tue September 5 2006 08:10, Eugene Roux wrote:
> > On Tue, 2006-09-05 at 07:37 -0500, Michael S. Zick wrote:
> > > On Tue September 5 2006 02:52, Eugene Roux wrote:
> > > >
> > > > Current Status:
> > > > * Multiple VEs are up and stable.
> > > > * VEs see their serial devices (all mapped to "/dev/modem").
> > > > * Connecting to "/dev/modem" using "cu -l" gives sane results.
> > > > * PPP connections establishes fine when tried individually.
> > > > * Trying to bring up more than one PPP interface at a time causes
> > > > the second to abort when it tries to set a default route.
> > > >
> > > > Any suggestions and/or tips?
> > > >
> > >
> > > man pppd
> > > See the: nodefaultroute and replacedefaultroute options.
> >
> > Hmm... 'nodefaultroute', used in combination with a bit 'ip rule'
> > hackery might be the answer here.
> >
> > Unfortunately it's been some years since I've done that: one forgets...
> > Time to stretch the mind again.
> >
> > > They should give you control of the trying to set a default
> > > route.
> > >
> > > You might check how many /dev/ppp0 devices are trying to
> > > be set up also. ppp usually makes the connection:
> > > ppp? <-> /dev/something
> >
> > This is correct, but fortunately :-
> >
> > > Since routing is on the host, you might have to poke at
> > > the ppp code to get it to take an option for the device
> > > name. I.E: pppdev=ppp0 (vserver 0), pppdev=ppp1 (vserver 1)
> >
> > This already happens. As far I can determine,
>
> That should be in your logs. You might have to set the
> debug option, I don't recall.
>
> > the second Vserver is
> > allocated the 'ppp1' interface as an instance of '/dev/ppp'.
> >
>
> Now that seems strange when you think about it...

Would be, but...

> As I understand your setup description, there is one instance of
> ppp running in each vserver (or will be) - so how does the vserver-02
> ppp instance see the vserver-01 ppp instance to know to increment
> the device name?

To get this to work, I had to turn off interface hiding. In fact I had
to turn it off to get PPP running at all, so the added advantage of
allowing PPP to increment interfaces was missed by me.

I put it down to all vservers sharing the same base kernel and that the
base kernel handled all the (real) networking...

> Since you describe each vserver as having its own /dev/modem then
> each vserver must also have its own /dev directory (correct?).

Quite correct, as it is. Each /dev/modem points at a different physical
interface but the /dev/ppp node is the same (ie. identical node numbers)
in all the vservers.

> I know less than nothing about the kernel portion of ppp but there
> might be something there that is breaking the isolation.

Daniel Hokka Zakrisson gave me the answer to unhiding the interfaces
from the vservers (amusingly enough the flower page will show you howw
to hide, but not how to *un*-hide) which allowed PPP to actually grab
its interface and set an IP address.

In other words, yes, the isolation had been thouroughly broken when it
comes to networking.

Was I going to provide hosting on these that would have had me waking up
at funny hours in cold shivers, but since I'm planning on using it for
dial-out tests and since I couldn't care less if the vservers can snoop
each others interfaces -- as long as they don't stomp on another
vserver's network, that is -- I consider all this within acceptable
parameters.

> Perhaps it does not recognize there are multiple instances of the
> userland portion running in isolation from each other.
>
> Since the kernel and presumably the kernel portion of ppp, is common...
> then internally, it should be maintaining some sort of alias information.
>
> That is;
>
> vserver-01:ppp0 <-> host:ppp1
> vserver-02:ppp0 <-> host:ppp2
> vserver-03:ppp0 <-> host:ppp3
>
> > Where it all comes crashing down is when the second ppp interface then
> > tries to set a default route and the first will -- quite understandably
> > I think -- not give it up util it's done.
> >
>
> That is also strange...
>
> Why would the first (or any) ppp instance in a virtual server be able to
> change the host's routing?

Ah, that would be because I gave it the capability...

        root@esm-vs01:~# cat /etc/vservers/vs01-vc01/bcapabilities
        CAP_SYS_TTY_CONFIG
        CAP_NET_ADMIN
        CAP_NET_RAW
        CAP_SYS_ADMIN
        CAP_IPC_LOCK
        CAP_IPC_OWNER
        root@esm-vs01:~#

Overkill, maybe, but desperate times...

Added to which, of course, the fact that it's being used as a probe and
not a server and that I'm planning on hardening the host seemed like ab
acceptable compromise to me.

> Sorry I can not answer the key question: "Is this broke or am I using it wrong?"

's cool. It does allow me to think through what I've been doing up until
now...

Regards,
        Eugéne

-- 
Eugéne Roux                "Fairy tales do not tell children the dragons
Cynical Romantic,                  exist.  Children already know dragons
Romantic Philosopher,              exist.  Fairy tales tell children the
Philosophising Cynic             dragons can be killed." G.K. Chesterton

_______________________________________________
Vserver mailing list
Vserver@list.linux-vserver.org
http://list.linux-vserver.org/mailman/listinfo/vserver

Received on Tue Sep 5 17:43:04 2006
[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Tue 05 Sep 2006 - 17:43:09 BST by hypermail 2.1.8