Intercept and Forward TCP 443 Traffic to an HTTPS Proxy (using CONNECT)
 

Hi,

I have a Linux router set up in a proxied environment, and I'm trying to have it intercept all IPv4 traffic directed at public addresses, then use a proxy to reach the destination.

I currently managed to do this for HTTP with TransProxy and iptables; all port 80 traffic is being sent to the transproxy daemon, which in turn creates the proxy requests for the proxy server. The replies are then sent back to the originating host as if it returned from the site requested by the user.

Most HTTPS proxies behave differently though; you need to use the CONNECT feature (RFC 2817). There's a program called stunnel that I'm trying to use, but what I'm trying to accomplish can't be done by it (when specifiying "protocol = connect", you must also specify "protocolhost" in the config file, i.e. the server that needs to be reached. The thing is, I want it to take the IP from the packet's destination as the protocolhost).


Any ideas? I can elaborate further if necessary.

I'm pretty sure there is no way to transparently proxy HTTPS. Like you are aware, that CONNECT has to occur for a proxy. The proxy where I am now generates it's own site certificates from an internal CA, so I get a fully signed and trusted mail.google.com cert from our internal CA so it can scan HTTPS traffic. But you still need a CONNECT. Transparent proxies are pretty horrible though, can I urge you to scrap that and just use a proper explicit proxy config?

In my environment, the proxy does not modify the traffic. It is possible with stunnel, but you must define each site separately (you can't use wildcards). So what happens now is that if you try to access site X, stunnel will access the proxy, but would have to request the site from its configuration file. I tried using "destination" as a wildcard in protocolHost, but what I saw later was that it simply requested "CONNECT destination" from the proxy.

right yes. You seem to understand the situation, sooooo?

So I was wondering if someone has an alternate solution, because I'm not a programmer and can't modify stunnel that drastically.

well you can't modify stunnel at all... you NEED that CONNECT method, so the browser NEEDS to know it's talking to a proxy.

The whole point is not having the browser know that it's talking to a proxy.
When using CONNECT, you have a raw socket between you and the server.

User opens https://www.google.com
DNS says www.google.com is 1.2.3.4
User sends 1.2.3.4:443 SYN
stunnel connects to the 192.168.1.1 proxy, and performs CONNECT 1.2.3.4:443
replies from the proxy are sent back to the user as if they originated from 1.2.3.4 and not 192.168.1.1.

Oh right, so I missed a bit there, you want stunnel to add the SSL? lots of sites work differently with and without SSL, but past that, you need to CONNECT to the domain name, not the IP. That's the name that's used to verify the certificate in the first instance, so at that stage in the process you've no idea what domain you want the certificate for anyway. So that's not going to work unless you drop the SSL cert on the floor, in which case, why bother??

That's the thing, the proxy doesn't check what data resides in the traffic, so it won't verify certificates at all.

I don't need stunnel to add SSL, I need it to encapsulate the traffic between the user and the server, through the proxy.

Ugh, we'll get there... so you can't just proxy based on a SYN, you've not got a *host* to connect to, just an IP. You don't get the hostname until after the SSL handshake is required, and you need the hostname to begin it in the first place. It's just not possible currently. There are new mechanisms specified for a client to request a site before SSL but they aren't in use yet.

Why can't you just do things normally and correctly and use the proxy directly??

I disagree. As an unproxied user, when you want to open an SSL resource, you still open a connection to IP:443, and the handshake (including HTTP GET hostname) occurs afterwards. The proxy doesn't _have_ to know the hostname.

Remember, the proxy isn't the originator of any data, only the user's browser and the web site are.

I can do things normally, but as you know, especially in the Linux world, there's no solution to make EVERYTHING use the proxy, and some programs aren't proxy-enabled. The biggest offenders would be in-package downloaders (e.g. adobe-flashplugin-installer for Ubuntu).

Hmm, I see what you mean, I'm sure there's some way this would break the SSL handshake though. I can't see a comparison of the flow with and without a CONNECT, but I'd think there must be a difference of some form. But then the two aren't actually interlinked. CONNECT is a generic mechanism...

Hang on, if you just base the CONNECT on the TCP syn, you'll lose the original destination at the point of redirecting it into stunnel. How do you know where to proxy onwards to? You'd need some additional mechanism to record the original packet details.