On Sat, Feb 26, 2011 at 8:12 AM, Bruno Prémont
<bonbons@linux-vserver.org> wrote:
> On Thu, 24 February 2011 Tim Spriggs <tims@uahirise.org> wrote:
>
>> On Thursday, February 24, 2011, Jeff Jansen <jeff.jansen@kkoncepts.net> wrote:
>> > On Friday 25,February,2011 08:39 AM, Tim Spriggs wrote:
>> >> This script was written for my environment so it assumes the use of
>> >> "vlan" interfaces of the form:
>> >>
>> >> vlan2/vlan3/...vlanN
>> >>
>> >> instead of the other forms.
>> >
>> > If someone would not mind helping me with my ignorance :-) what's the
>> > advantages of using vlans in your vserver networking? I've always used
>> > "novlandev" (which is the default now, I think) in networking and simply
>> > created the addresses as secondaries on the interface.
>> >
>> > We use tagged vlans on our switches, so I understand the concept. But
>> > I'm having trouble seeing what I gain from putting every vserver on its
>> > own vlan. I think I'm missing something conceptually.
>> >
>> > For folks who are using vlans in their vserver networking, I'd
>> > appreciate knowing why you choose to go that route and how you feel it
>> > improves your setup as opposed to simply creating secondary addresses on
>> > the interfaces.
>> >
>> > TIA
>> >
>> > Jeff Jansen
>> >
>>
>> I am new to multiple routing tables so please step in and correct me
>> if anyone sees something wrong.
>>
>> We have several networks with different security levels and we want to
>> put server on machines that are on multiple networks. For the sake of
>> simplicity I will explain what I am solving in a 2-vlan scenario while
>> the machines we have setup are still a bit more complex than this.
>>
>> On vlan1 we are running some network (let's just say 192.168.1.0/24)
>> and on vlan2 we are running another network (similarly
>> 192.168.2.0/24). The way we have our network topology configured,
>> hosts on vlan2 can contact hosts on vlan1 and hosts on vlan1 are
>> effectively DMZ hosts that can provide services to the world. So,
>> traffic can be initiated from vlan2->vlan1 but not from vlan1->vlan2.
>>
>> Let's imagine we have some large computer that will host many services
>> on both vlans but each service itself needs to be on only one network.
>> Again for simplicity, imagine vserver1 on vlan1 (192.168.1.10) and
>> vserver2 on vlan2 (192.168.2.10). With the normal ip stack and routing
>> setup vserver1 will be able to send packets onto vlan2 because as the
>> packet is formed and sent into the network stack, the Linux kernel
>> will say, "hey, I have a route directly to that network from this
>> other interface. Let's just use it." This creates several problems:
>>
>> 1) traffic goes where you don't want it.
>> 2) when hosts in vlan2 attempt to connect to vserver1, the server will
>> respond with it's usual ACK but the ACK will go out of the wrong
>> interface... as will all other traffic even if this is OK on your
>> network
>> 2.1) This is not ok on a network where the boundary between vlan1 and
>> vlan2 is a stateful firewall because the fw never sees the ACK
>> 2.2) This is bad anyway because it's not what you should expect
>>
>> Now for the part I haven't tested yet. Communication from
>> vserver1->vserver2 (dmz->internal) should not work without contacting
>> the firewall first. While this is not optimal in terms of bandwidth,
>> it is needed for our security model.
>>
>> Enter multiple routing tables and ip rules (thanks again for the
>> pointer Bertl.) With ip rules you can specify that traffic originating
>> from vlan1 (192.168.1.0/24) should use a routing table specially
>> tailored for delivering packets correctly out of the vlan1
>> interface... or without the "knowledge" of the vlan2 network. This is
>> where I get lazy in my script; I assume that vlanN corresponds to the
>> routing table id=N. Now we can state with confidence that traffic
>> coming from vserver1 (which *is* a secondary IP on the vlan1
>> interface) will only "know" about the vlan1 network and the vlan1
>> default gateway.
>>
>> --
>>
>> Next problem: IPv6.
>>
>> We have deployed IPv6 on our networks and as such we need this
>> configuration to work with IPv6 if it is going to work at all. The
>> problem and significant difference between IPv4 and IPv6 in our
>> scenario is that we use SLAAC to configure the default gateway for all
>> of our hosts. This means we can not hard-code the default gateway and
>> then just add that to our secondary routing tables. The second bump is
>> that when our hosts are coming up and the interfaces are being
>> plumbed, they may not know what the default router on the network is
>> (or even their own network prefix for that matter) so the script needs
>> to wait for that information to become available and then create the
>> appropriate ip rule and ip route entries.
>>
>> My solution was to fork and literally wait (via polling) for
>> information to appear on the device in question. Once it does then the
>> fork continues and propagates the routes into the secondary routing
>> tables.
>>
>> I hope this clears things up a little but if not I can try to make
>> them more muddy by explaining more ;)
>
> Beware that with IPv6 you might need to trigger rerouting of your packets
> in order to make sure they go out on the right interface.
> To do this, e.g. set a marker with iptables for traffic from your guest
> (this way netfilter will cause kernel to recalculate routing for your
> packets). The reason is that for IPv6, in opposition to IPv4 kernel first
> tries to determine routing and only in a second time selects source
> address, thus it may select a route that looks better for your target but
> for which the source address is not available to your guest. Then when
> determining your source address it will for an address from your guest but
> maybe route through the wrong interface (under IPv6 IP addresses are
> system wide, e.g. only very loosely bound to interfaces).
>
> Because of your security consideration it looks like you might be better
> of attempting to combine linux-vserver and mainline net namespaces. (no
> idea if it is possible or how hard it is [to implement]) At least it would
> provide the cleanest subnet separation.
>
> Bruno
Hi Bruno,
Thanks for this pointer. It may be necessary to implement...
As it turns out, even with the config mentioned above, vserver1 can
connect to vserver2 if it is on the same machine. I'm not sure if
there is a way to turn the short-circuiting of IPs off in this
configuration but it would be nice if it exists. :)
-Tim
Received on Mon Feb 28 20:51:51 2011