Packets not entering FORWARD chain

first of all, this is not an every-day routing issue. The setup is fairly complex, so let me state it before.

I got a router with, lets keep it simple, 3 interfaces. eth0, eth1, eth2. eth2 is used for pppoe. eth0 & eth1 have the clients.

Okay so far so good, all basic.. Now here comes the tricky thing: I create a bunch of macvlan-interfaces on top of eth0 and eth1, the name schema is:

g1eth0 : g1 for gate1, eth0 indicates on what physical interface its laying on

This I got for every uplink I provide, lets say 3, 1 pppoe and 2 VPNs. These are then merged into bridges named after the gate.

So far we got these interfaces:

<iface>:<description>
eth0   : our 1st subnet is here
eth1   : our 2nd subnet is here
eth2   : our pppoe is hooked here
ppp0   : our pppoe uplink
tap0   : our vpn1 uplink
tap1   : our vpn2 uplink
g1eth0 : advertised gate over uplink1 on clients in eth0
g1eth1 : advertised gate over uplink1 on clients in eth1
g2eth0 : advertised gate over uplink2 on clients in eth0
g3eth1 : advertised gate over uplink3 on clients in eth1
gate1  : bridge containing g1eth0 and g1eth1
gate2  : bridge containing g2eth0
gate3  : bridge containing g3eth1

As I said, a bunch of interfaces... Notice that an uplink can be advertised over several physical interfaces, thats why we got the bridges.

Alright now lets take a look at the routing rules:

32763:  from all fwmark 0x3 lookup 202
32764:  from all fwmark 0x2 lookup 201
32765:  from all fwmark 0x1 lookup 200

Okay this is not so spectacular, obviously, it only checks what FWMARK a pkg has and pushes it to the according table.

The routing tables:

200: default via 1.2.3.4 dev ppp0 src 4.3.2.1

201: default via 5.6.7.8 dev tap0 src 8.7.6.5

202: default via 9.10.11.12 dev tap1 src 12.11.10.9

Okay the IPs are just for to fill the gaps, you should be familiar with the syntax ;)

Right now we got the routing tables, routing rules and the interfaces - but we're missing out the pkg marking, so this is being done in iptables:

iptables -t mangle -A PREROUTING -i gate1 -s 10.0.0.0/16 -j MARK --set-xmark 0x1/0xffffffff
iptables -t mangle -A PREROUTING -i gate2 -s 10.0.0.0/16 -j MARK --set-xmark 0x2/0xffffffff
iptables -t mangle -A PREROUTING -i gate3 -s 10.0.0.0/16 -j MARK --set-xmark 0x3/0xffffffff

Okay for explanation, we mark all pkgs comming in our bridges with the right value for the routing rules.

Now I also had to do some tweaks in arp_announce and arp_ignore so that the right MAC is being advertised for the g*eth*-interfaces. This post is getting rather full, so I will skip describing it, both are set to 2.

The filter:FORWARD chain is empty for now, it just logs the pkgs it gets.

Now NAT'ing: iptables -t nat -A POSTROUTING -s 10.0.0.0/16 -j MASQUERADE.

All default policies for iptables are ACCEPT.

tcpdump shows that the incomming pkgs are directed to the right MAC according to the g*eth*-interfaces.

mangle:PREROUTING counters for the rules increment as they should.

ip_forward verified to be 1.

filter:FORWARD counters are NOT incrementing.

I got LOG rules in every chain, but the pkgs seem to vanish once passed the mangle:PREROUTING.

Any ideas why?

Addition I: I placed a TRACE rule in PREROUTING as the comment suggested me, ironically it doesn't show any of the pings my clients are running.

Addition II: After some playing around with the rules,tracing,promisc,... I noticed that I see the data getting in on ethX but not on gateX. Seems like the brigde-interface is just dropping it, no wonder the kernel cant get it into forward.

Why does my bridge-interface do this?

bridge name     bridge id               STP enabled     interfaces
gate1           8000.dead000200b5       no              g1eth0
                                                        g1eth1

It could be blocked due to reverse path filtering.

Try turning it off: http://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.kernel.rpf.html