A question about routing, and iptables in linux...

I have 2 internet connections and a linux box that sits on both connections. I would like to route traffic over each of these connections via traffic type. This should be pretty straight forward application of routing and iptables but I'm hitting a brick wall and I can't seem to find any solutions.

So here's the basic configuration.

• Network 1 (N1): 192.168.1.0/24 and the router is 192.168.1.1
• Network 2 (N2): 192.168.0.0/24 and the router is 192.168.0.1

Default gateway on the linux machine is N2 via 192.168.0.1.

If I add a static route to use N1 via 192.168.1.1 for a specific host this works fine (ie: route add -host www.reddit.com gateway 192.168.1.1)

But I want to route on traffic type. So lets say I want 443 to go out N1 instead of the default N2. These are the steps based on my understanding...

• Create a separate routing table like so...

  /sbin/ip route flush table 4

  /sbin/ip route show table main |grep -Ev ^default | while read ROUTE; do ip route add table 4 $ROUTE; done

  /sbin/ip route add table 4 default via 192.168.1.1

• Mark the traffic destined for 443...

  /sbin/iptables -t mangle -A PREROUTING -p tcp --dport 443 -j MARK --set-mark 4

  /sbin/iptables -t mangle -A OUTPUT -p tcp --dport 443 -j MARK --set-mark 4

• Add a routing rule to cause that traffic to use the newly created routing table...

  /sbin/ip rule add fwmark 4 table 4

  /sbin/ip route flush cache

• This almost worked. The traffic was going out the right interface, but had the wrong source address. So I SNAT'd it....

  /sbin/iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source 192.168.1.2

  /sbin/iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 192.168.0.2

• Now traffic goes out, the connection is made, packets come back (as evidenced via tcpdump) but it seems the packets never reach the client app. They're either clanked in the kernel, or someplace else and never reach the app.

I have a feeling I'm missing something pretty basic here but I'm not sure what. Has anybody done this and can shed some light on what I'm missing?

This is a fresh ubuntu 10.04 install. For testing purposes I've removed apparmor and ufw. iptables is wide open and all policies are set to ACCEPT.

Ideas? Thanks!

Edit: Formatting

Edit2: I've been doing more testing. SYN packet makes it out the correct interface. The SYNACK comes back and everything looks right in wireshark, but it seems the box just ignores it. I cranked up logging on iptables and it doesn't even see the packet. Looks like it's getting dropped in the stack somewhere before iptables.