LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 04-10-2012, 05:58 AM   #1
wgualla
LQ Newbie
 
Registered: Aug 2009
Posts: 4

Rep: Reputation: 1
iptable mangle + ip rule fwmark + masquerade lost packets


Hi, can anyone help me determine why I lose packets?

I have configured multiple VPNs to different customers having subnet overlapping (client 1, subnet 192.168.0.0/24 via vpn tunnel on ppp0; client 2, subnet 192.168.0.0/24 via vpn tunnel on ppp1 ), my solution is to assign two different subnets on the lan interface (client 1 192.168.100.0/24, client 2 192.168.101.0/24), mark packets with iptables mangle and one routing table for each mark as follows:

# Customer # 1:
iptables -t nat -I POSTROUTING -o ppp0 -j MASQUERADE
iptables -t mangle -I PREROUTING -i eth0 \
-d 192.168.100.0/24 -j MARK --set-mark 100
ip rule add fwmark 100 table 100
ip route add default dev ppp0 table 100


# Customer # 2:
iptables -t nat -I POSTROUTING -o ppp1 -j MASQUERADE
iptables -t mangle -I PREROUTING -i eth0 \
-d 192.168.101.0/24 -j MARK --set-mark 101
ip rule add fwmark 101 table 101
ip route add default dev ppp1 table 101

# Note: ppp0 has ip addr 192.168.180.7
# ppp1 has ip addr 192.168.180.8


ok, so far, so good, BUT I LOSE PACKETS!!!!:

trying from my lan machine (192.168.33.152: ping 192.168.100.251) packets arrives to router; looking at output interface I can see request & reply echo packets, but somewhere on linux connection tracking system, reply packets lose (I can see echo request packets on eth0, request & replay on ppp0, but reply packet doesn't arrives output interface eth0)

NOTE: disabling ip rule fwmark and using default main route table, everything works fine.


# tcpdump -nn -q -i eth0 icmp
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
11:33:01.019693 IP 192.168.33.152 > 192.168.100.251: ICMP echo request, id 768, seq 38145, length 40
11:33:06.207066 IP 192.168.33.152 > 192.168.100.251: ICMP echo request, id 768, seq 38401, length 40

# tcpdump -nn -q -i ppp0
listening on ppp0, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
11:56:18.942009 IP 192.168.180.7 > 192.168.0.251: ICMP echo request, id 768, seq 40449, length 40
11:56:18.994315 IP 192.168.0.251 > 192.168.180.7: ICMP echo reply, id 768, seq 40449, length 40
11:56:24.259976 IP 192.168.180.7 > 192.168.0.251: ICMP echo request, id 768, seq 40705, length 40
11:56:24.312380 IP 192.168.0.251 > 192.168.180.7: ICMP echo reply, id 768, seq 40705, length 40


some additional information:

#uname -ar
Linux router 2.6.27-17-generic #1 SMP Fri Mar 12 03:09:00 UTC 2010 i686 GNU/Linux

router:~# lsb_release -d
Description: Ubuntu 8.10



thank everyone in advance.
 
Old 04-11-2012, 03:54 AM   #2
nikmit
Member
 
Registered: May 2011
Location: Nottingham, UK
Distribution: Debian
Posts: 178

Rep: Reputation: 34
See http://www.linuxquestions.org/questi...debian-936957/ and the current thread by donalbane about ppp default route. No solution yet but I think the discussions might be related.

Other than that if you provide a sketch of what your layout looks like it will make it easier to provide a relevant answer.

One thing that doesn't look right to me is at this point is how you mark the packets destined for one specific IP address and then ping a different IP. How is your NAT set up?
To set this up with overlapping networks you will need a) the gateway on the client side doing port address translation, so you can route all traffic to their network with a static route for one or a few /32 ip addresses or b) devise a fairly complex NAT solution on your router, possibly by doing SNAT on the client facing interfaces and then marking on the internet facing interface based on the NAT-ed source address.

Nik
 
Old 04-11-2012, 04:43 AM   #3
wgualla
LQ Newbie
 
Registered: Aug 2009
Posts: 4

Original Poster
Rep: Reputation: 1
Hi Nik, thank for your answer.
You're right, I forgot include nat commands. Sorry. I have use iptables NETMAP to translate each subnet.
iptables -I PREROUTING -d 192.168.100.0/24 -i eth0 -j NETMAP --to 192.168.0.0/24
iptables -I PREROUTING -d 192.168.101.0/24 -i eth0 -j NETMAP --to 192.168.0.0/24


Let me explain the situation with a little schema:

Code:
Quote:
+-----------------+ | | ping 192.168.100.251 reply from ORION | my pc +----+ ping 192.168.101.251 reply from HERCULES |192.168.33.152 | | | | | +----- AT THIS POINT echo reply has gone. +-----------------+ | | |<-------+ | +------------------+-----------+ iptables -d 192.168.100.0/24 -> --set-mark 100 ### PREROUTING ### | eth0 | iptables -d 192.168.101.0/24 -> --set-mark 101 | 192.168.33.10 | iptables -mark 100 -d 192.168.100.0/24 -> netmap --to 192.168.0.0/24 | | iptables -mark 101 -d 192.168.101.0/24 -> netmap --to 192.168.0.0/24 | ppp0 ppp1 | |192.168.180.7 192.168.180.8 | iptables -t nat -o ppp0 -j MASQUERADE ### POSTROUTING ### +-----+-------------------+----+ iptables -t nat -o ppp1 -j MASQUERADE | |<---------------------+ | table 100 | table 101 | | fwmark 100 | fwmark 101 | | default dev ppp0 | default dev ppp1 +-- AT THIS POINT I'VE echo request & reply !!!! | | | | | | +--------+--------+ +-------+---------+ | pppX | | pppY | | 192.168.180.107 | | 192.168.180.108 | | | | | | 192.168.0.1 | | 192.168.0.1 | | eth0 | | eth0 | +---+-------------+ +---+-------------+ | | | | | ORION | HERCULES | +-------------+ | +-------------+ +-+192.168.0.251| +-+192.168.0.251| | +-------------+ | +-------------+ | | . . | +-------------+ | +-------------+ +-+192.168.0.250| +-+192.168.0.250| | +-------------+ | +-------------+ | +-------------+ | +-------------+ +-+192.168.0.249| +-+192.168.0.249| +-------------+ +-------------+
I hope this be helpful, I look forward to your suggestions.

Thank again.

Last edited by wgualla; 04-11-2012 at 05:11 AM.
 
Old 04-11-2012, 05:25 AM   #4
nikmit
Member
 
Registered: May 2011
Location: Nottingham, UK
Distribution: Debian
Posts: 178

Rep: Reputation: 34
I suppose you have no other way of doing the addressing, or it is not up to you. Everything would be much simpler if the subnets don't overlap.

If you have to stick with it, the easiest way would be to do what all ISPs do - provide an address at pppY and pppX and conigure it so all traffic is masqueraded to that address.
Code:
          |                   |
 +--------+--------+  +-------+---------+
 |      pppX       |  |      pppY       | iptables -t nat -A postrouting -o pppY -j MASQUERADE
 | 192.168.180.107 |  | 192.168.180.108 | iptables -t nat -A postrouting -o pppX -j MASQUERADE
 |                 |  |                 |
 |  192.168.0.1    |  |  192.168.0.1    |
 |  eth0           |  |  eth0           |
 +---+-------------+  +---+-------------+
     |                    |
At the moment you have all 4 addresses in your 2 ppp pairs in the same 192.168.180.0/24 subnet, so the kernel sees them as connected to the same network.
Either reduce the subnet size, or change the network number on one of them. So pppX becomes 192.168.181.107 and ppp0 192.168.181.7
Code:
          |                   |
 +--------+--------+  +-------+---------+
 |      pppX       |  |      pppY       | iptables -t nat -A postrouting -o pppY -j MASQUERADE
 | 192.168.181.107 |  | 192.168.180.108 | iptables -t nat -A postrouting -o pppX -j MASQUERADE
 |                 |  |                 |
 |  192.168.0.1    |  |  192.168.0.1    |
 |  eth0           |  |  eth0           |
 +---+-------------+  +---+-------------+
     |                    |
I would expect things to work with just the above, and no further marking or NATing.

Your entire diagram would be
Code:
|                 |  default gateway 192.168.33.10    
| my pc           +----+ 
|192.168.33.152   |    |
|                 |    | 
+-----------------+    | 
                       |
                       |
    +------------------+-----------+ 
    |                 eth0         | 
    |             192.168.33.10    | 
    |                              | Once ppp0 and ppp1 are in different subnets, and orion and hercules NATed,
    |    ppp0            ppp1      | nothing should be needed here other than basic forwarding permissions.
    |192.168.181.7  192.168.180.8  | 
    +-----+-------------------+----+ 
          |                   |
          |                   | 
          |                   |
   box1   |                   | box2
 +--------+--------+  +-------+---------+
 |      pppX       |  |      pppY       | iptables -t nat -A postrouting -o pppY -j MASQUERADE (on box2)
 | 192.168.181.107 |  | 192.168.180.108 | iptables -t nat -A postrouting -o pppX -j MASQUERADE (on box1)
 |                 |  |                 |
 |  192.168.0.1    |  |  192.168.0.1    |
 |  eth0           |  |  eth0           |
 +---+-------------+  +---+-------------+
     |                    |
     |                    |
     |   ORION            |   HERCULES    default gateway 192.168.0.1 (for all hosts in both networks)
     | +-------------+    | +-------------+
     +-+192.168.0.251|    +-+192.168.0.251|
     | +-------------+    | +-------------+
     |                    |
     .                    .
     | +-------------+    | +-------------+
     +-+192.168.0.250|    +-+192.168.0.250|
     | +-------------+    | +-------------+
     | +-------------+    | +-------------+
     +-+192.168.0.249|    +-+192.168.0.249|
       +-------------+      +-------------+
 
Old 04-12-2012, 06:33 PM   #5
wgualla
LQ Newbie
 
Registered: Aug 2009
Posts: 4

Original Poster
Rep: Reputation: 1
Hi Nik, thank again for your reply.

your suggestion would be fine if I wanted to go from Hercules to my pc but the reality is just the opposite.

Hercules is a TerminalServer and I would like to logme in on it (the same is true for Orion, and a lot of other machines, this is just an example).

All my clients have they own ip range selection and subnet definition, I've no way to change it (this is the root of the challenge, they were here for a long time before).

Beyond the theological issues of the case, my problem is that somewhere in the nine circles of connection tracking system my packages are lost on their way back and I suspect it is by the use of different routing tables of the main. (the problem persists even removing all the configuration settings relative to the box2/pppY)

if you still have temerity to continue dealing with this issue, I look forward to your comments.

Thanks again.
 
Old 04-13-2012, 06:35 AM   #6
nikmit
Member
 
Registered: May 2011
Location: Nottingham, UK
Distribution: Debian
Posts: 178

Rep: Reputation: 34
OK I think I know what the task is now, and I got it working in a test set up. Hopefully this will work for you:

Code:
iptables -t mangle -A PREROUTING -i eth0 -d 192.168.100.0/24 -j MARK --set-mark 0x1111
iptables -t mangle -A PREROUTING -i eth0 -d 192.168.101.0/24 -j MARK --set-mark 0x2222
iptables -t nat -A PREROUTING -i eth0 -d 192.168.100.0/24 -j NETMAP --to 192.168.0.0/24
iptables -t nat -A PREROUTING -i eth0 -d 192.168.101.0/24 -j NETMAP --to 192.168.0.0/24
ip rule add fwmark 0x1111 table 2
ip rule add fwmark 0x2222 table 3
ip route add 0.0.0.0/0 dev ppp0 table 2             # if needed you can make the route more specific here
ip route add 0.0.0.0/0 dev ppp1 table 3             # and here
ip route flush cache
This is working with icmp on ethernet rather than ppp interfaces. There is a thread here describing problems with default routes and ppp interfaces, see how you get on and post
 
Old 04-13-2012, 06:58 AM   #7
wgualla
LQ Newbie
 
Registered: Aug 2009
Posts: 4

Original Poster
Rep: Reputation: 1
Thank you again Nik, all you write is correct and works fine... theorically, but we had two day working around. The key point was to disable the rp_filter (Reverse Path filtering).


SOLVED AT LAST!!!!!!!!!!


echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter

Last edited by wgualla; 04-13-2012 at 07:15 AM.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] iptable rule amartlk Linux - Newbie 2 12-18-2011 11:36 PM
Multi-WAN Problem with IPROUTE2/IPTABLES - Packets disappear between MANGLE & NAT alpharomeo31 Linux - Kernel 2 10-18-2011 10:12 AM
iptable how many rule iptable can manage toure32 Linux - Networking 1 05-13-2010 05:34 AM
iptable rule vinaytp Linux - Newbie 1 10-26-2009 02:39 AM
Same Iptable rule to be avoided? Santoshkb Linux - Networking 2 12-21-2007 06:55 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 05:03 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration