[vserver] Avoiding kernel internal routing among vserver clients

From: Thomas Weber <l_vserver_at_mail2news.4t2.com>
Date: Tue 31 Jul 2007 - 01:56:10 BST
Message-Id: <1185843370.5190.147.camel@localhost>

Hello all,

this seems to be a tricky configuration I haven't found information
about (it's also difficult to make up a good Subject for this).

Anyway, the Problem:
H1 H2 are Hosts,
VS1 VS2 VSx are vservers on these hosts.

The Setup is like this:

NetA VS2 VS4 VS6
                  eth0 eth0
-GW--------------- H1 ----------- H2 -----------
                  eth1 eth1
NetB VS1 VS3 VS5

The GW is the regular connection between NetA and NetB.
Vservers are hosted on either H1 or H2 (Hx if you like).

I'd like to have all traffic from vservers in NetA to NetB pass the GW.
The Problem is that Hx routes the traffic between it's vservers
internally (if VS1 and VS2 are hosted on H1, traffic between VS1/2 never
hits the GW).

Right now I've partially solved the problem by using the netfilter ROUTE
target (
http://www.netfilter.org/documentation/HOWTO//netfilter-extensions-HOWTO-4.html#ss4.5)
by 'putting' packets from the lo Interface onto the ethx Interface where
I want them. So far so good. On the receiving side i 'get' them from the
ethx interface and 'put; them back onto the lo interface again, so that
the kernel can answer them.

So this is my test setup:
host:
eth0 192.168.1.52/24, gw .254
eth1 192.168.2.52/24, gw .254
vs test1:
eth0 192.168.1.152/24
vs test2:
eth1 192.168.2.152/24

~# iptables -t mangle -L -v -n
Chain PREROUTING (policy ACCEPT 18206 packets, 1551K bytes)
 pkts bytes target prot opt in out source destination
 5991 503K ROUTE 0 -- eth0 * 192.168.2.152 192.168.1.0/24 ROUTE oif:lo
[...]
Chain POSTROUTING (policy ACCEPT 10881 packets, 1452K bytes)
 pkts bytes target prot opt in out source destination
 6759 568K ROUTE 0 -- * lo 192.168.2.152 192.168.1.0/24 ROUTE oif:eth1

this is only to test pinging test1 from test2 right now.
tcpdump on the GW shows me that the echo requests passing the firewall.
I can also see these echo request approaching the host on eth0.
tcpdump -i eth0 -n icmp:
02:47:14.752500 IP 192.168.2.152 > 192.168.1.152: ICMP echo request, id 28948, seq 251, length 64

BUT tcpdump -i lo -n icmp gives me:
02:43:30.752294 IP 192.168.2.152 > 192.168.1.152: ICMP echo request, id 28948, seq 27, length 64
02:43:30.752358 IP 192.168.1.152 > 192.168.1.52: ICMP echo reply, id 28948, seq 27, length 64
                                           ^^^^^
which is the wrong address to reply to. Is this because of the vserver
kernel patches or 'normal' Linux behaviour?

Does any of you have an idea how to solve the initial Problem? If there
is a better way then the iptables ROUTE target i'd be more than happy to
try this.

The Versions:
debian etch
util-vserver 0.30.212-1
self compiled linux-2.6.21.6-vs2.2.0.3 + netfilter ROUTE

tia,
  Tom
Received on Tue Jul 31 01:56:38 2007

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Tue 31 Jul 2007 - 01:56:43 BST by hypermail 2.1.8