Background: I work in a "disconnected" environment. We have our internal network running a few Windows machines and a whole horde of RHEL7/8 servers. There's a firewall at the edge of that network, and on the other side of it is a larger organizational network with Internet access. In the past we've had holes in that firewall for internal servers to go through if there's something on the Internet they absolutely need.
The situation: My coworkers wanted to have a proxy server to have all our internal systems that need to touch the internet go through. So we set up Squid on a RHEL8 server, gave it a list of source IPs it was allowed to receive from, and a whitelist of destinations those IPs were allowed to go to on the Internet. The proxy sits outside the firewall, and only those machines that need Internet access have holes to go through to the proxy. And it works wonderfully, except for a single case...
The problem: We have a server that we want to use to host an internal repository of Python packages (using python-pypi-mirror. It is set up to reach through the proxy (via the firewall) to PyPI. I can run
wget https://python.org and it connects successfully and downloads the index page. But when I run pip...
Code:
$ pip3 install --user python-pypi-mirror
Collecting python-pypi-mirror
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff41f142f98>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/python-pypi-mirror/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff41f142ac8>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/python-pypi-mirror/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff41f142710>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/python-pypi-mirror/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff41f142dd8>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/python-pypi-mirror/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7ff41f142400>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/python-pypi-mirror/
Could not find a version that satisfies the requirement python-pypi-mirror (from versions: )
No matching distribution found for python-pypi-mirror
On the Squid proxy, the logs show the client connects and reaches through to PyPI (TCP_TUNNEL/200), but then we get TCP_TUNNEL_ABORTED. Googling suggests there may have once been a problem where libraries that pip depends on forced all HTTP connections to use HTTPS, does anyone know if that still happens? If not, does anyone have any other ideas?
Python 3 pip cannot reach PyPI through squid proxy
