Hi. I'm jon.404, a Unix/Linux/Database/Openstack/Kubernetes Administrator, AWS/GCP/Azure Engineer, mathematics enthusiast, and amateur philosopher. This is where I rant about that which upsets me, laugh about that which amuses me, and jabber about that which holds my interest most: *nix.
OpenBSD AWS BGP VPN
In an earlier blogpost, I covered configuring a static AWS VPN setup with OpenBSD. I promised I'd revisit it to include the BGP (dynamically routed) version if I ever got it working.
Hi, I got it working.
**sorta** working (major emphasis on "sorta").
Let me explain...OpenBSD is a policy-based IPSEC engine. AWS operates on a route-based IPSEC engine. The major difference, in laymen's terms, is that with route-based IPSEC you can configure a tunnel, say, between two public IPs that share a set of aliases (two 169.254.X.Y addresses on the same /30). You can then setup a route to point *other* traffic through that tunnel's interface, say, to 192.168.0.0/24 from 10.0.0.0/8. You need one IPSEC tunnel (two data security associations) to do this (and, with AWS, *two* tunnels (four data security associations), in a master/backup configuration).
Policy-based IPSEC is simpler. If the kernel has a route to a network, utilize it. Cisco routers (in particular) are notorious for configuring dozens (or more) security association pairs for remote networks when a single aggregated route could do the job just fine (convincing your average Cisco Certified Security guru to aggregate tunnels and control traffic via ACLs, however, is an entirely different story that I dare not recount (the repeated times I've had to have that conversation) here =). OpenBSD's IPSEC engine routes traffic through enc0, which isn't an interface type that you can *route* traffic through (that I've figured out, at least).
With route-based IPSEC (in AWS's config, at least), BGP keepalives are used to determine if a tunnel is up, and if so, utilize it (via injected routes). If one tunnel fails, route traffic via the other tunnel. Simple. You can give the tunnels separate path-lengths, so they operate in a master/failover setup, or the same path-length so they operate master/master (which can lead to asymmetric routing and other fun stuff). (It really gets fun when your customer realizes that their CGW is a single point of failure, so they setup two CGW's (on physically separate routers), each with two tunnels to separate AWS endpoints (running via two separate DX connections), all with slightly different path-lengths so BGP routes traffic in (master/master)/(failover/failover). Whew. If that ain't Enterprise, I dunno what is).
With policy-based IPSEC (again, AWS's config, at least), you have to bring up another IPSEC tunnel if the other dies.
The problem arises, however, that bringing up a new IPSEC tunnel isn't an instantaneous event. Traffic that is waiting could timeout if the tunnel takes too long to come up. Moreover, waiting for a new tunnel to come up defeats the purpose of BGP (alter routes when a down path is discovered).
Ok, with that said, I have dual IPSEC tunnels to two separate 169.254.X.Y tunnels to AWS on my OpenBSD firewall at home (and a similar config on pfsense running on a VPS). BGP routes are exchanged across these tunnels (BGP does the job of SLA monitor nicely in such configurations). Unfortunately, talking to the VGW 169.254.X.Y addresses is not exactly useful, so for a policy-based configuration like this, I have to have another VPN tunnel configured to accomplish reaching my VPC...and it doesn't dynamically move when BGP updates occur! That's right...if the *useful* tunnel dies, you have to manually move it (manually or "automagically" via cron'd script, I suppose) rather than simply rely on BGP injected routes to get you where you need to go. Pfft!
So yes, it can be done. But it's sorta like (to quote a movie I once saw) a "poopie-flavored lollypop". You could do it, but WHY? By going this route (haha...sigh), you gain two additional IPSEC tunnels that aren't really useful for anything, and your "production" tunnel traffic is still manually failed over. Not that useful.
I can share the configs if anyone is interested.
Hi, I got it working.
**sorta** working (major emphasis on "sorta").
Let me explain...OpenBSD is a policy-based IPSEC engine. AWS operates on a route-based IPSEC engine. The major difference, in laymen's terms, is that with route-based IPSEC you can configure a tunnel, say, between two public IPs that share a set of aliases (two 169.254.X.Y addresses on the same /30). You can then setup a route to point *other* traffic through that tunnel's interface, say, to 192.168.0.0/24 from 10.0.0.0/8. You need one IPSEC tunnel (two data security associations) to do this (and, with AWS, *two* tunnels (four data security associations), in a master/backup configuration).
Policy-based IPSEC is simpler. If the kernel has a route to a network, utilize it. Cisco routers (in particular) are notorious for configuring dozens (or more) security association pairs for remote networks when a single aggregated route could do the job just fine (convincing your average Cisco Certified Security guru to aggregate tunnels and control traffic via ACLs, however, is an entirely different story that I dare not recount (the repeated times I've had to have that conversation) here =). OpenBSD's IPSEC engine routes traffic through enc0, which isn't an interface type that you can *route* traffic through (that I've figured out, at least).
With route-based IPSEC (in AWS's config, at least), BGP keepalives are used to determine if a tunnel is up, and if so, utilize it (via injected routes). If one tunnel fails, route traffic via the other tunnel. Simple. You can give the tunnels separate path-lengths, so they operate in a master/failover setup, or the same path-length so they operate master/master (which can lead to asymmetric routing and other fun stuff). (It really gets fun when your customer realizes that their CGW is a single point of failure, so they setup two CGW's (on physically separate routers), each with two tunnels to separate AWS endpoints (running via two separate DX connections), all with slightly different path-lengths so BGP routes traffic in (master/master)/(failover/failover). Whew. If that ain't Enterprise, I dunno what is).
With policy-based IPSEC (again, AWS's config, at least), you have to bring up another IPSEC tunnel if the other dies.
The problem arises, however, that bringing up a new IPSEC tunnel isn't an instantaneous event. Traffic that is waiting could timeout if the tunnel takes too long to come up. Moreover, waiting for a new tunnel to come up defeats the purpose of BGP (alter routes when a down path is discovered).
Ok, with that said, I have dual IPSEC tunnels to two separate 169.254.X.Y tunnels to AWS on my OpenBSD firewall at home (and a similar config on pfsense running on a VPS). BGP routes are exchanged across these tunnels (BGP does the job of SLA monitor nicely in such configurations). Unfortunately, talking to the VGW 169.254.X.Y addresses is not exactly useful, so for a policy-based configuration like this, I have to have another VPN tunnel configured to accomplish reaching my VPC...and it doesn't dynamically move when BGP updates occur! That's right...if the *useful* tunnel dies, you have to manually move it (manually or "automagically" via cron'd script, I suppose) rather than simply rely on BGP injected routes to get you where you need to go. Pfft!
So yes, it can be done. But it's sorta like (to quote a movie I once saw) a "poopie-flavored lollypop". You could do it, but WHY? By going this route (haha...sigh), you gain two additional IPSEC tunnels that aren't really useful for anything, and your "production" tunnel traffic is still manually failed over. Not that useful.
I can share the configs if anyone is interested.
Total Comments 0
