Re: [vserver] Partly solved: Avoiding kernel internal routing among vserver clients

From: Thomas Weber <l_vserver_at_mail2news.4t2.com>
Date: Wed 08 Aug 2007 - 21:19:04 BST
Message-Id: <1186604345.25125.93.camel@localhost>

Am Mittwoch, den 08.08.2007, 17:19 +0200 schrieb Herbert Poetzl:

> the kernel implementation does not differenciate between
> IPs on eth0 and IPs on eth1, they are configured on the
> host, so they are _local_ and not very surprisingly, the
> traffic will neither use eth0 nor eth1 but lo instead

Yep. If you look at it from the host. From inside the vserver the view
is a bit different.

> > As a sidenote, I could very well imagine that there are setups where
> > people put their private/internal vservers on an interface of their
> > own without realizing that their public vservers can reach the private
> > one without any restrictions.
>
> if there are no restrictions (think iptables) then local
> IPs can communicate (this is not different in any way
> from what you get on every Linux box)

Again. People think about virtualized Servers and don't want to worry
about the hosting System. Technically and regarding the host, you're
right.

> > > if you go for a completely virtualized network stack
> > > (mainline is working on that already) and do not mind
> > > the larger overhead in resources and the drastically
> > > increased traffic on your DMZ network, as well as the
> > > lower network performance (virtualization here has
> > > quite noticeable overhead too) instead for lightweight
> > > IP isolation (that is what Linux-VServer is doing),
> > > then you can get your setup where all traffic (even
> > > naturally host local traffic' is routed to the
> > > gateway and back again ...
> >
> > Where can I find more information about this?
> > Once upon a time there was this networking-ng stuff, which was
> > supposed to do something thelike.
>
> yep, we made a working prototype and figured that
> the introduced overhead is not really acceptable
> for a generic solution. shortly after that mainline
> discovered virtualization and since then they are
> working on a virtualized stack ...

ahhh... _that_ mainline ;-)

> I would say a setup which does S/DNAT on output and
> input from/to local IPs should do the trick ...
>
> when I find some time, I might try it myself ...

Actually, ahem as I just figured, the ROUTE module of iptables does the
trick quite nice. Except for icmp it seems...
*/me banging his head against the wall*
Earlier, I've only tried ping which doesn't work for some reason. But
all your mails gave me the impression that it's supposed to work already
with the usual stuff that's already there.

Remember the setup I described:
eth0 192.168.1.52/24, gw .254
eth1 192.168.2.52/24, gw .254
vs test1:
eth0 192.168.1.152/24
vs test2:
eth1 192.168.2.152/24

# iptables -t mangle -L -v
Chain PREROUTING (policy ACCEPT 5079 packets, 410K bytes)
 pkts bytes target prot opt in out source destination
   45 2566 ROUTE 0 -- eth1 any 192.168.1.152 192.168.2.0/24 ROUTE oif:lo
  139 8956 ROUTE 0 -- eth0 any 192.168.2.152 192.168.1.0/24 ROUTE oif:lo

Chain POSTROUTING (policy ACCEPT 3082 packets, 487K bytes)
 pkts bytes target prot opt in out source destination
   59 3406 ROUTE 0 -- any lo 192.168.1.152 192.168.2.0/24 ROUTE oif:eth0
  183 12652 ROUTE 0 -- any lo 192.168.2.152 192.168.1.0/24 ROUTE oif:eth1

TCP and UDP connections between test1 and test2 seem to work as i want
it (everything via the GW).
ICMP is broken:
(ping from test2 to test1 running)
# tcpdump -n -i lo
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
22:07:19.874175 IP 192.168.2.152 > 192.168.1.152: ICMP echo request, id 56342, seq 16, length 64
22:07:19.874228 IP 192.168.1.152 > 192.168.1.52: ICMP echo reply, id 56342, seq 16, length 64
                                           ^^^^^
of course it can't work that way.
The Packets arrive just fine on eth0 with correct source address. I
don't know why the answer gets targeted to the hosts address.

Now is this something possibly caused by the vserver patches or is the
kernel itself to blame?

2.6.21.6-vs2.2.0.3

thanks for your patience (and vserver of course)
  Tom
Received on Wed Aug 8 21:19:34 2007

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Wed 08 Aug 2007 - 21:19:37 BST by hypermail 2.1.8