Python3 - <urlopen error [SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:997)>

Turbocapitalist · 07-05-2023, 12:09 AM

I am getting the following error,

Code:

<urlopen error [SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:997)>

with the following script,

Code:

#!/usr/bin/python3                                                              

import urllib.request

url = 'http://www.imy.se/en/news/rss/'

try:
    with urllib.request.urlopen(url) as response:
        try:
            print("Fetch succeeded")
        except:
            print("Fetch failed")

except urllib.request.URLError as e:
    print(e)
    exit(1)

I'm not so familiar with TLS. Is the problem on the server side? If so, is there a relevant CVE to point the webmaster to?

pan64 · 07-05-2023, 01:47 AM

here you can find some ideas: https://stackoverflow.com/questions/...ation-disabled

NevemTeve · 07-05-2023, 01:50 AM

Try this:

Code:

url = 'https://www.imy.se/en/news/rss/'

Turbocapitalist · 07-05-2023, 01:59 AM

Quote:

Originally Posted by NevemTeve

Try this:

Code:

url = 'https://www.imy.se/en/news/rss/'

Yes, same result.

NevemTeve · 07-05-2023, 02:10 AM

Now this:

Code:

openssl s_client -connect www.imy.se:443

Edit: It does show the error for me (OpenSSL 3.1.1):

Code:

00472F8FEF7F0000:error:0A000152:SSL routines:final_renegotiate:unsafe legacy renegotiation disabled:ssl/statem/extensions.c:893:

Workaround:

Code:

openssl s_client -legacy_renegotiation -connect www.imy.se:443

or:

Code:

openssl s_client -legacy_server_connect -connect www.imy.se:443

In program, it is OP_LEGACY_SERVER_CONNECT option.

teckk · 07-05-2023, 11:04 AM

This is a hodge podge of your code and mine sorry.

Here:

Code:

import requests
import urllib3
import urllib.request
import ssl

class CustomHttpAdapter (requests.adapters.HTTPAdapter):
    def __init__(self, ssl_context=None, **kwargs):
        self.ssl_context = ssl_context
        super().__init__(**kwargs)

    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = urllib3.poolmanager.PoolManager(
            num_pools=connections, maxsize=maxsize,
            block=block, ssl_context=self.ssl_context)

def get_legacy_session():
    ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
    ctx.options |= 0x4
    session = requests.session()
    session.mount('https://', CustomHttpAdapter(ctx))
    return session
    
try:
    with get_legacy_session().get('https://www.imy.se/en/news/rss/') as responce:
        try:
            print("Fetch succeeded")
        except:
            print("Fetch failed")

except urllib.request.URLError as e:
    print(e)
    exit(1)

Code:

python ./a2.py
Fetch succeeded

Turbocapitalist · 07-06-2023, 04:22 AM

Thanks. The work-arounds are interesting but I'm much more interested in the cause, either on my end or on their end. It appears that the same problem may affect many sites.

teckk · 07-06-2023, 11:47 AM

The problem is openssl 3 and it's adoption or not. It is breaking things. openssl 3.1.1 broke some things for me.

Info:
https://wiki.archlinux.org/title/OpenSSL

If anyone wants to play. And maybe someone can say more about this, I'm simply studying it...

For me this is /etc/ssl/openssl.cnf

Code:

[openssl_init]
#Remark
#providers = provider_sect

#Add
ssl_conf = ssl_sect

#Add
[ssl_sect]
system_default = system_default_sect

#Add
[system_default_sect]
Options = UnsafeLegacyRenegotiation

#Add
[provider_sect]
default = default_sect

Code:

from urllib import request, error

url = 'https://www.imy.se/en/news/rss/'

agent = ('Mozilla/5.0 (Windows NT 10.0; Win64 x64; rv:109.0) '
            'Gecko/20100101 Firefox/115.0')

user_agent = {'User-Agent': agent,
            'Accept': 'text/html,application/xhtml+xml,'
            'application/xml;q=0.9,*/*;q=0.8',
            'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
            'Accept-Encoding': 'none',
            'Accept-Language': 'en-US,en;q=0.8',
            'Connection': 'keep-alive'}
            
try:
    req = request.Request(url, data=None, headers=user_agent)
    page = request.urlopen(req)
    
except error.HTTPError as e:
    print("Http error")
except error.URLError as e:
    print('Url Error')
except TypeError as e:
    print('Type Error')
except ValueError as e:
    print("Value error")

try:
    html = page.read().decode('utf-8', 'ignore')

except NameError:
    print('Name error')
    
print(html)

Ok, I thnk that I would do that with python-ssl per site and not system wide. Turbocapitalist, I'm trying to figure this out myself.

Edit:
Copied the wrong path. Corrected

teckk · 07-06-2023, 05:19 PM

I kept looking because I am interested in this myself. I now understand what/why.

This works for now. No need to do system wide change. It's actually a vulnerability that openssl 3 closed. Problem is that some servers needs to be updated. But, if you see more of them:

~/openssl.cnf

Code:

openssl_conf = openssl_init

[openssl_init]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
Options = UnsafeLegacyRenegotiation

Code:

export OPENSSL_CONF=~/openssl.cnf
python ./myscript.py
<urllib.request.Request object at 0x7f998351b150>
<http.client.HTTPResponse object at 0x7f9983524700>
<?xml version="1.0" encoding="utf-8"?><rss xmlns:a10="http://www.w3.org/2005/Atom" version="2.0"><channel><title>Integritetsskyddsmyndigheten - News in English</title><link>http://www.imy.se/en/news/rss/</link><description>Integritetsskyddsmyndigheten - News in English</description><item><guid isPermaLink="false">935d07a1-64f7-4183-9e3d-12c5f95a193e</guid><link>https://www.imy.se/en/news/four-companies-must-stop-using-google-analytics/</link><title>Four companies must stop using Google Analytics </title><description>The Swedish Authority for Privacy Protection (IMY) has audited how four c
...

And, for what you are wanting, probably

Code:

import xml.etree.ElementTree

Or something else that will parse xml.

teckk · 07-06-2023, 05:41 PM

And, I could not stand to NOT parse that, now that I get it.

Code:

export OPENSSL_CONF=~/openssl.cnf

Code:

#!/usr/bin/python

from xml.etree import ElementTree
from urllib import request

#Make a user agent string for urllib to use
agent = ('Mozilla/5.0 (Windows NT 10.1; Win64; x64; rv:109.0) '
        'Gecko/20100101 Firefox/115.0')
        
user_agent = {'User-Agent': agent}

class MakeList():
    def __init__(self, url, fname):
    
        #Get the xml to parse
        req = request.Request(url, data=None, headers=user_agent)
        html = request.urlopen(req)
        tree = ElementTree.parse(html)
        root = tree.getroot()
        
        #Get tag data
        tagA = root.findall('./channel/item/title')
        tagB = root.findall('./channel/item/link')
        tagC = root.findall('./channel/item/description')
        tagD = []
        
        #Append lines with separator
        for a,b,c in zip(tagA,tagB,tagC):
            tagD.extend([a.text, b.text, c.text, '_' * 70])
            
        #Print
        for i in tagD:
            print(i)
                
        #Write list to file
        with open((fname), 'a') as f:
            for line in tagD:
                f.write('%s\n' % line)
                
if __name__ == "__main__":
    
    #Urls, log names
    A = ('https://www.imy.se/en/news/rss/', 'Imy.log')
        
    B = ('file:///home/path/file.xml', 'Imytest.log')
    
    #Choose rss here
    url, fname = A

    MakeList(url, fname)

Code:

python ./mytest.py
Four companies must stop using Google Analytics 
https://www.imy.se/en/news/four-companies-must-stop-using-google-analytics/
The Swedish Authority for Privacy Protection (IMY) has audited how four companies use Google Analytics for web statistics. IMY issues administrative fines against two of the companies. One of the companies has recently stopped using the statistics tool on its own initiative, while IMY orders the other three to also stop using it.
______________________________________________________________________
Administrative fee against Spotify 
https://www.imy.se/en/news/administrative-fee-against-spotify/
The Swedish Authority for Privacy Protection (IMY) has audited how Spotify handles customers' right to access their personal data. The deficiencies that have been discovered cause IMY to issue an administrative fine of SEK 58 million against the company.
______________________________________________________________________
Data protection officers point to problems applying GDPR 
https://www.imy.se/en/news/data-protection-officers-point-to-problems-applying-gdpr/
A survey by the Swedish Authority for Privacy Protection (IMY) notes that less than half of responding data protection officers assess that their own organisation works continually and systematically with data protection.
______________________________________________________________________
...

Love it when members ask questions about problems. Makes me learn myself.