[SOLVED] wget2 OCSP response too old and stapled OCSP failed

walecha · 10-09-2021, 06:35 AM

Wget2 with openssl does not perform OCSP verification if the server certificate does not contain an OCSP URI and then the connection or download fails

Code:

$ wget2 https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz
[0] Downloading 'https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz' ...
HTTP response 302  [https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz]
Adding URL: https://github-releases.githubusercontent.com/5101141/e4612500-eca9-11e8-8306-c58af06c65f5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211009%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211009T101044Z&X-Amz-Expires=300&X-Amz-Signature=12f57c7b85eaf9c11b2fad1507d2d3cf08630c0920a9bd15c8ccbd958550f254&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5101141&response-content-disposition=attachment%3B%20filename%3Djq-1.6.tar.gz&response-content-type=application%2Foctet-stream
OCSP response is too old. Ignoring.
Could not verify stapled OCSP response. Aborting.
Could not complete TLS handshake: invalid status response
OCSP response is too old. Ignoring.
Could not verify stapled OCSP response. Aborting.
Could not complete TLS handshake: invalid status response
OCSP response is too old. Ignoring.
Could not verify stapled OCSP response. Aborting.
Could not complete TLS handshake: invalid status response
OCSP response is too old. Ignoring.
Could not verify stapled OCSP response. Aborting.
Could not complete TLS handshake: invalid status response
Failed to connect (101)
Failed to connect (101)
Failed to connect (101)
Failed to connect (101)
Failed to connect: Connect error

Here is debug level 2 from stock wget2 (build with openssl)

Code:

[0] Downloading 'http://2' ...
09.174926.882 cookie_create_request_header for host=2 path=(null)
09.174926.882 has 20.205.243.166:443
09.174926.882 trying 20.205.243.166:443...
09.174926.883 OpenSSL initialized
09.174926.883 Sending 'status_request' extension in handshake
09.174926.883 ALPN offering h2
09.174926.883 ALPN offering http/1.1
09.174926.883 No cached TLS session available. Will run a full handshake.
09.174927.171 No HPKP pinning found for host 'github.com'
09.174927.171 OCSP URI not given and not found in certificate. Skipping OCSP check for cert 0.
09.174927.171 OCSP URI not given and not found in certificate. Skipping OCSP check for cert 1.
09.174927.171 No HPKP pinning found for host 'github.com'
09.174927.171 No HPKP pinning found for host 'github.com'
09.174927.171 No stapled OCSP response was received. Continuing.
09.174927.172 Handshake completed (full handshake - not resumed)
09.174927.172 TLS session discarded
09.174927.172 ALPN: Server accepted protocol 'h2'
09.174927.172 established connection github.com

Rebuilding wget2 using gnutls instead of openssl (--with-ssl=gnutls --with-openssl=off) causes wget2 to still perform OCSP verification even though the server certificate does not contain an OCSP URI. In my case wget2 was using the digicert ocsp server. Connect/download completed without error.

Code:

[0] Downloading 'http://2' ...
09.175126.553 cookie_create_request_header for host=2 path=(null)
09.175126.554 has 20.205.243.166:443
09.175126.554 trying 20.205.243.166:443...
09.175126.554 GnuTLS init
09.175126.566 GnuTLS system certificate store is empty
09.175126.566 Certificates loaded: 142
09.175126.566 GnuTLS init done
09.175126.566 TLS False Start requested
09.175126.566 ALPN offering h2
09.175126.566 ALPN offering http/1.1
09.175126.846 host has no pubkey pinnings stored in hpkp db
09.175126.846 host has no pubkey pinnings stored in hpkp db
09.175126.846 host has no pubkey pinnings stored in hpkp db
09.175126.846 host has no pubkey pinnings stored in hpkp db
09.175126.846 resolving ocsp.digicert.com:80...
09.175126.846 has 117.18.237.29:80
09.175126.846 trying 117.18.237.29:80...
09.175126.865 # sent 291 bytes:
POST / HTTP/1.1
Host: ocsp.digicert.com
Accept-Encoding: identity
Accept: */*
Connection: close
Content-Type: application/ocsp-request
Content-Length: 127

marav · 10-09-2021, 07:13 AM

according to : ./configure --help in the source pkg, we can use --with-openssl option

Code:

blackstar :: source/n/wget2 ‹master*› » diff -u wget2.SlackBuild wget2.SlackBuild.new                                                                                                    1 ↵
--- wget2.SlackBuild    2021-09-13 06:36:47.000000000 +0200
+++ wget2.SlackBuild.new        2021-10-09 14:09:46.059355243 +0200
@@ -94,7 +94,7 @@
   --mandir=/usr/man \
   --infodir=/usr/info \
   --docdir=/usr/doc/$PKGNAM-$VERSION \
-  --with-ssl=openssl \
+  --with-openssl \
   --disable-static \
   --build=${ARCH}-slackware-linux || exit 1

works for me

Code:

blackstar :: ~ » wget2 https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz                    
[0] Downloading 'https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz' ...
HTTP response 302  [https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz]
Adding URL: https://github-releases.githubusercontent.com/5101141/e4612500-eca9-11e8-8306-c58af06c65f5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211009%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211009T121227Z&X-Amz-Expires=300&X-Amz-Signature=57eaac5f9443e1cc4e56d8bf23bf10d7f1cbd205dec7ce00590d0a2087f9016b&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5101141&response-content-disposition=attachment%3B%20filename%3Djq-1.6.tar.gz&response-content-type=application%2Foctet-stream
[0] Downloading 'https://github-releases.githubusercontent.com/5101141/e4612500-eca9-11e8-8306-c58af06c65f5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211009%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211009T121227Z&X-Amz-Expires=300&X-Amz-Signature=57eaac5f9443e1cc4e56d8bf23bf10d7f1cbd205dec7ce00590d0a2087f9016b&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5101141&response-content-disposition=attachment%3B%20filename%3Djq-1.6.tar.gz&response-content-type=application%2Foctet-stream' ...
Saving 'jq-1.6.tar.gz'
HTTP response 200  [https://github-releases.githubusercontent.com/5101141/e4612500-eca9-11e8-8306-c58af06c65f5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211009%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211009T121227Z&X-Amz-Expires=300&X-Amz-Signature=57eaac5f9443e1cc4e56d8bf23bf10d7f1cbd205dec7ce00590d0a2087f9016b&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5101141&response-content-disposition=attachment%3B%20filename%3Djq-1.6.tar.gz&response-content-type=application%2Foctet-stream]

Code:

blackstar :: ~ » ls -l jq-1.6.tar.gz 
-rw-r--r-- 1 marav marav 1750584 nov.  20  2018 jq-1.6.tar.gz

it seems to have been reported upstream
https://gitlab.com/gnuwget/wget2/-/issues/563

marav · 10-09-2021, 05:26 PM

https://github.com/rockdaboot/wget2/issues/243

walecha · 10-09-2021, 09:28 PM

Thank you for the info. I've looked for that bug report but couldn't find it. So the culprit is the selected ssl library to link to because of --with-ssl=openssl. Instead of using -lssl, configure using -lopenssl. That is indeed an upstream bug.

marav · 10-10-2021, 05:48 PM

fixed

Code:

Sun Oct 10 22:20:01 UTC 2021
n/wget2-2.0.0-x86_64-2.txz:  Rebuilt.
  Rebuilt using --with-openssl. Thanks to walecha and marav.

marav · 10-26-2021, 04:24 PM

Update from github :

Code:

--with-openssl explicitly selects the crypto library for checksumming etc. 
The TLS library to use is auto-selected in this case.
While --with-ssl=openssl explicitly selects the TLS library to be OpenSSL.

So depending on which dev libraries you installed, both builds may use a different TLS library (or different versions) - 
e.g. GnuTLS is preferred over OpenSSL in automatic mode.

It would be interesting to see what you end up with. If you could share the config.log for both cases, I can find out.

marav · 01-05-2022, 03:00 AM

FYI:

juaristi commented:

Quote:

OCSP response is too old. Ignoring.

This error comes from the fact that the stapled OCSP response is older than 3 days.
I don't know why the check fails in OpenSSL and succeeds in GnuTLS, have to look into that.
In the meanwhile, could you disable that with --no-ocsp-date? That should disable this particular check.

but

Quote:

Hi,

Curiously, the issue ( --with-openssl) no longer exists ...

Code:

blackstar :: ~ » wget2 -V
GNU Wget2 2.0.0 - multithreaded metalink/file/website downloader

+digest +https +ssl/openssl +ipv6 +iri +large-file +nls -ntlm -opie +psl -hsts
+iconv +idn2 +zlib +lzma +brotlidec +zstd +bzip2 +lzip +http2 +gpgme

Code:

blackstar :: ~ » wget2 https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz
[0] Downloading 'https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz' ...
HTTP response 302  [https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz]
Adding URL: https://objects.githubusercontent.com/github-production-release-asset-2e65be/5101141/e4612500-eca9-11e8-8306-c58af06c65f5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220105%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220105T082800Z&X-Amz-Expires=300&X-Amz-Signature=c7d164fc079b38162a8afc8689283ababa7e0e8b376f17f6d2dff27957dc8df3&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5101141&response-content-disposition=attachment%3B%20filename%3Djq-1.6.tar.gz&response-content-type=application%2Foctet-stream
[0] Downloading 'https://objects.githubusercontent.com/github-production-release-asset-2e65be/5101141/e4612500-eca9-11e8-8306-c58af06c65f5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220105%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220105T082800Z&X-Amz-Expires=300&X-Amz-Signature=c7d164fc079b38162a8afc8689283ababa7e0e8b376f17f6d2dff27957dc8df3&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5101141&response-content-disposition=attachment%3B%20filename%3Djq-1.6.tar.gz&response-content-type=application%2Foctet-stream' ...
Saving 'jq-1.6.tar.gz.1'
HTTP response 200  [https://objects.githubusercontent.com/github-production-release-asset-2e65be/5101141/e4612500-eca9-11e8-8306-c58af06c65f5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220105%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220105T082800Z&X-Amz-Expires=300&X-Amz-Signature=c7d164fc079b38162a8afc8689283ababa7e0e8b376f17f6d2dff27957dc8df3&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5101141&response-content-disposition=attachment%3B%20filename%3Djq-1.6.tar.gz&response-content-type=application%2Foctet-stream]

marav · 01-05-2022, 03:02 AM

Anyway, he pushed a fix

https://gitlab.com/gnuwget/wget2/-/c...c0b1a13b31b9fc