Hi guys,

Our ISP provides us with Layer 2 hardware (Modem?/Switch? (Hatteras)) for our leased line and internet access. I have been told that I should set up a VLAN capable router to separate the internet traffic from the internal traffic.

OK lets google...

I found that linux is capable of VLAN routing. Nice!

I have setup opensuse 10.3 put in two NICs and did the following

vconfig add eth0 10
vconfig add eth0 20
ifconfig public.ip.add.ress netmask 255.255.255.252 eth0.10 up
ifconfig 192.168.0.1 netmask 255.255.255.0 eth0.20 up

plugged this NIC into the HATTERAS hardware (with a straight cable), and thought that this way I should be able to ping the public gateway or any ip out on the internet.
Well, no.
My ISP is telling me that I should create a VLAN trunk to be able to 'use the internet', but as I understand in linux if I create any number of VLANs on one NIC they are already trunked. I also got the info, that the traffic is tagged, and I can separate the traffic reading the tags. I already read that some NICs are not able to VLAN because they are not able to handle the increased packet size, and also that the MTU setting is important (dono' the exact value though, only that its important)

I thought that a linux machine can act as a router and firewall in such a case, because proprietary switches/routers use linux as embedded os.

this is my first meeting with VLAN so if this whole post does not make any sense or you think that I just need to RTFM more then tell me!

P.S.: I also have some (3) Dlink 3226 Layer2 switches around, but I think it would be waste to use a 24 port switch on this subject.

thanks and regards
tom

generally this seems reasonable, but let's have some standard info... output of ifconfig, vconfig, arp -n and route -n please.

you're using tags 10 and 20... are these right? You've not explictly confirmed that you're aware of the right tags.

thanks for the response.
First of all, I am not aware of any tags (the ones I am using I just made up, concentarting on not to use id 1), the ISP guys just left us with this modem thing without any info, and the information I have, comes from numerous phone calls to their helpdesk. They told me that because they provide a Layer 2 switch, so every configuration is on me.

I have changed a couple of settings because to two sites are 50 kms away from each other, so for now I have given 192.168.157.249 to the other nic, so I can reach it from the main site, and given 192.168.2.1 to the vlan10 so it does not interfere with our current 192.168.0.0 and 192.168.157.0 setup.


here is the requested info:

--------- IFCONFIG -------------------------------
eth0 Link encap:Ethernet HWaddr 00:14:85:24:A6:25
inet addr:192.168.157.249 Bcast:192.168.157.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:271 errors:0 dropped:0 overruns:0 frame:0
TX packets:39 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:29621 (28.9 Kb) TX bytes:1750 (1.7 Kb)

eth1 Link encap:Ethernet HWaddr 00:E0:7D:E6:BF:65
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:89609 errors:0 dropped:0 overruns:0 frame:0
TX packets:3033 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:11163366 (10.6 Mb) TX bytes:181980 (177.7 Kb)
Interrupt:16

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:3054 errors:0 dropped:0 overruns:0 frame:0
TX packets:3054 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:341828 (333.8 Kb) TX bytes:341828 (333.8 Kb)

vlan10 Link encap:Ethernet HWaddr 00:E0:7D:E6:BF:65
inet addr:192.168.2.1 Bcast:192.168.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

vlan20 Link encap:Ethernet HWaddr 00:E0:7D:E6:BF:65
inet addr:###.###.###.### Bcast:###.###.###.### Mask:###.###.###.###
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
--------- IFCONFIG END ----------------------------------

Here is the output of vconfig, but this does not seem anything useful

------------ VCONFIG ------------------------------------
Expecting argc to be 3-5, inclusive. Was: 1

Usage: add [interface-name] [vlan_id]
rem [vlan-name]
set_flag [interface-name] [flag-num] [0 | 1]
set_egress_map [vlan-name] [skb_priority] [vlan_qos]
set_ingress_map [vlan-name] [skb_priority] [vlan_qos]
set_name_type [name-type]

* The [interface-name] is the name of the ethernet card that hosts
the VLAN you are talking about.
* The vlan_id is the identifier (0-4095) of the VLAN you are operating on.
* skb_priority is the priority in the socket buffer (sk_buff).
* vlan_qos is the 3 bit priority in the VLAN header
* name-type: VLAN_PLUS_VID (vlan0005), VLAN_PLUS_VID_NO_PAD (vlan5),
DEV_PLUS_VID (eth0.0005), DEV_PLUS_VID_NO_PAD (eth0.5)
* bind-type: PER_DEVICE # Allows vlan 5 on eth0 and eth1 to be unique.
PER_KERNEL # Forces vlan 5 to be unique across all devices.
* FLAGS: 1 REORDER_HDR When this is set, the VLAN device will move the
ethernet header around to make it look exactly like a real
ethernet device. This may help programs such as DHCPd which
read the raw ethernet packet and make assumptions about the
location of bytes. If you don't need it, don't turn it on, because
there will be at least a small performance degradation. Default
is OFF.
-------------- VCONFIG END --------------------------------------

Here is the output of arp -n

--------------- ARP -N -----------------------------------------
Address HWtype HWaddress Flags Mask Iface
192.168.157.253 ether 00:30:48:75:EF:36 C eth0
----------- ARP END --------------------------------------------

At last here is route -n
---------------- ROUTE -N --------------------------------------
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
###.###.###.### 0.0.0.0 255.255.255.252 U 0 0 0 vlan20
192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 vlan10
192.168.157.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 192.168.157.254 0.0.0.0 UG 0 0 0 eth0
------------ ROUTE END -----------------------------------------------

hope this helps

tom

I wouldn't rely on VLANS to seperate internal and external trafic, nor would I share a single switch between any networks/interfaces that were not in the same network.. (Internal/External/DMZ)

VLAN INSECURITY
VLANS WERE CREATED TO ISOLATE LANS, BUT NOT FOR THE PURPOSES OF SECURITY
http://www.spirit.com/Network/net0103.html

I would put a dual NIC Firewall betweek the Internet connection (Public) and the Internal Network (Private).

I have no idea who your ISP is, but personally I think they gave you very bad advice.

"I wouldn't rely on VLANS to seperate internal and external trafic, nor would I share a single switch between any networks/interfaces that were not in the same network.. (Internal/External/DMZ)

VLAN INSECURITY
VLANS WERE CREATED TO ISOLATE LANS, BUT NOT FOR THE PURPOSES OF SECURITY
http://www.spirit.com/Network/net0103.html

I would put a dual NIC Firewall betweek the Internet connection (Public) and the Internal Network (Private).

I have no idea who your ISP is, but personally I think they gave you very bad advice."

Thats my idea also. At least I think, because the linux router I am trying to setup up has two NICs (or can have even more).
First NIC should have the public IP and some internal IP (with VLAN), and the second NIC should have our interal network IP.
The idea behind this is that I do not need a VPN connection between the two sites like now, but they can/could/should be in the same internal network, without anybody being able to crack the network, because we are phisically wired together through our ISP. And we only have actually one internet connection at one of our sites, and everything is routed through that router/modem/whatever. And of course I am concerned about our network security, but right now it seems more important to get the whole thing working.
Right now we are sitting on a 3 Mb/384K ADSL line at one site, and a Synchron 1 Mb/s microwave connection at the other site, which gives us poor performance, between the two sites. The leased line is 99,5% guaranteed 2 Mb/s and 6 Mb/s connection synchron.

bye
tom

Ah OK I mistakenly got the impression from your first post, you were trying to make a router-on-a-stick with vlans on a single switch sharing the internal and external networks..

Sorry for the missunderstanding.

Thanks everyone for the reponses, but this case is finally solved.
Our ISP 'forgot' to tell me the VLAN IDs they have set up. Now that I know the correct ID changed my config to that, at it works perfectly.

Sorry for not getting back sooner... but yeah you can't guess / pick a vlan id like that.

It's pretty common to business connections to be presented as layer 2 connections, seen plenty over the last few years with intrasite links and external connections being trunked on 802.1q

towme, I'm very interested in your solution. I'm facing a similar issue.

Our ISP offers a symmetrical, Hatteras based, product. We are looking to move to it. However, I've realized there's a problem. Presently we have 3 separate networks that use DSL connections. We're looking to move them all to sharing one, faster, enterprise connection. What they proposed we do is just plop our existing firewalls behind the new Hatteras 'modem' on a DMZ switch and give them their new IP info and away we go. The problem with this is that I MUST do some QoS/traffic shaping here to make sure that our VoIP system gets priority access and gets a reserved share of the bandwidth when it needs it. Even though the Hatteras products have built in functions to do this, our ISP does not offer that programming.

Now as far as I know, (I'm your basic 'jack of all trades, master of a few' type of sysadmin so I'm no CCNE), if you want to manage bandwidth with this type of setup, you need to do it at Layer 2, not Layers 3&4.

Do your VLAN's let you do that?

Does your setup give you the ability to do QoS, traffic shaping and prioritizing per outside IP or port?

IMO, this should all be done on the Hatteras box, not with another box stuck in between. However, if your solution works, I'm certainly willing to slap together a Linux router and make it dance.

Sorry if it seems like I'm hijacking this thread. but it seems like I'm facing nearly the same problem as the OP.

Thanks,
klgit

Sorry Klgit I have to disappoint you, I do not use traffic shaping yet, because we do not really use anything that would require high bandwidth between our sites. VOIP is only in test phase, our accounting system is a thin client system so it can run over 36k line even, the domain controllers synchronize over the night (so I don't care how slow it goes)

We do not get any access to the Hatteras modems, so anything I need to do must be done on the router(s).

Formerly we had a dsl and a microwave connection but this is clearly a better solution. We have a whole bunch of net based utilities and portals we have to work with. And no one is complaining anymore that during the day the connection would be slow or unreliable.

Right now I am using Vyatta as a firewall at the site where we have the actual internet connection. I have to tell you I have mixed feelings about it. It is definitely not straight forward, but they claim they are as good as a cisco solution. and they have good documentation, and not to mention that I found only them who supported VLANs out of the box.

Now that I have spent a week with the setup, it seems pretty easy and straight forward, and if I need to add a site then it could be done in half an hour, because you do not need to set up a different firewall for each site

Our road warriors can reach either site through one VPN connection.

So all in all I can really recommend this solution for anybody. Because from the administration point of view its just a lot easier and simpler.

tom

towme,

Thanks for the reply.

Well, you'll absolutely need traffic shaping once your VoIP goes live, especially if you are sharing bandwidth. Also, remote terminal sessions are surprisingly sensitive to bandwidth management. Though in their case it's latency, not speed that's the issue.

Same here, ISP has us locked out. Even worse they don't offer to program the Hatteras at all. No additional QoS features or anything other than basic connectivity.

That's good to hear. That's our primary reason for switching off DSL as well. Our ISP swears that the Hatteras based service is more reliable and consistent. Luckily, the other local businesses I've talked to that have the service confirm that.

I've been seriously considering Vyatta as well, however, without knowing for sure it's going to work for our application, I'm not too keen on spending the money on them. Plus the free edition removes the feature I'm interested in. The easy setup GUI.
A fellow sysadmin also highly recommended Astaro. They licensed that and built their own router PC's to run it on out of good but unused older server hardware.

This is our problem now. We have Cisco firewall's and VPNs at each remote connecting to a PIX at the main site now. The biggest issue for me is we do not control them. We get them from a service provider (ADP) and they charge every time you talk to them.

I'm working on a long term plan to replace them all with something else. Probably Vyatta. One thing that impressed me about Vyatta is that they told me they can convert our existing Cisco setups to their equipment. In other words, port the programming over so we can drop in the Vyatta routers.

Yeah, we pay per seat for this now, I'd like to have total control of this system. Plus I don't charge myself to make changes.

Glad to hear it. It relieves a little stress to hear from the Hatteras users that it's apparently a solid system. At least there will be something that I have to worry about less.

Meanwhile, I did a lot of research over the weekend regarding my bandwidth management question. I found what I think will be an ideal solution for us in the short term. I thought I'd share it in case anyone else is interested. I'm going to test it this week or weekend hopefully. It's called Zeroshell (see http://www.zeroshell.net/eng/ ).
Here's the things that I really like so far.

- Linux based
- Lightweight, built for the job.
- powerful but simple GUI option.
- Pre-built images to install to compact flash.
- Supports exactly the bandwidth management options I'm looking for.
- Works in transparent bridge mode so I don't need to change anything else on the network.
- supports DPI (deep packet inspection) so I can catch and manage streams that can't be or are difficult to handle at the TCP or UDP level like VoIP.
- Free

This page is the one that got me really excited about it since it basically is the instructions for setting up exactly what I want to do on our network.
http://www.zeroshell.net/eng/qos/

Hope this helps someone else stuck in the same boat.

I'll try and post results.

Tom, I'm also interested in hearing follow ups on your progress and any tips or gotchyas you find.


Thanks again,
KLGIT

To update, we went ahead with the Hatteras service and have been fairly happy with it so far.

As for the bandwidth management, I did what any good IT admin would do. I built my own solution. I got a fan-less mini-PC and installed a Linux based router OS (ZeroShell) on a CF card.

It works quite well, though I wish ZeroShell were more customizable than it is, but it's still the best solution I found for what I'm doing with it (bandwidth management).
I could have rolled my own custom Linux solution, but honestly I have too much other work to do to take the time. Plus I like the GUI/WEB interface that ZeroShell has.

Anyway, the short version is that you can do bandwidth management yourself. In fact I'm even considering replacing our Cisco router(s) entirely and give the ZeroShell box their jobs.