[SOLVED] [QuteBrowser] URL-redirection works but not in QuteBrowser??

Michael Uplawski · 03-08-2022, 03:00 PM

Good evening

Preliminary remarks:

1. This is *not* a network-question, as you will see.
2. I have removed .local/share/qutebrowser/webengine prior starting QuteBrowser and then have also executed :reload -f.

For years, I have had a working redirection from my domain
uplawski.eu
to
www.uplawski.eu

The redirection appears to still be valid to about everybody I ask and I can confirm with Firefox or W3M.

In QuteBrowser, however, the URL https://uplawski.eu produces the welcome-page of my hosting association « L'AutreNet ». In their help newsgroup / mailing-list, nobody could reproduce my observations, so far. The admins confirm the active redirection.

For the time I must assume that nothing is wrong with the redirection, but I cannot understand what QuteBrowser does.

This does not make sense, does it?

computersavvy · 03-08-2022, 08:57 PM

It does make sense depending on the actual config.
Does it actually get redirected to another site, or is it simply an alias in dns for the host name. Does the web server properly return the www name with every request? Is the www site a virual web site or is it the primary site on that host?

What I suspect is that the QuteBrowser is reporting the host address requested and does not switch to the host address replied by the server, possibly caused by not actually switching to a different physical host address.

ondoho · 03-08-2022, 11:51 PM

Maybe you have to tell QuteBrowser to allow redirections? MAybe it's default setup is very security conscious?

Michael Uplawski · 03-09-2022, 06:52 AM

Quote:

Originally Posted by computersavvy

It does make sense depending on the actual config.
Does it actually get redirected to another site, or is it simply an alias in dns for the host name.

This is an alias, i.e. a redirection that I configured for an ill-designated “subdomain” uplawski.eu, whilst www.uplawski.eu is the locally hosted “subdomain”. I get the impression that the terminology is arbitrary. The domain, which is managed by my registrar – not identical with Web- and Mail-host – is also just “uplawski.eu”.
Edit: Wait a moment... I will have to check my configuration in the Registrar's configuration interface. But it should be just like I wrote, above.

OKAY: Let us assume that I do not know anything. This is what I see:

1) Host WWW, type A, 80.67.160.70

The (Web-)Hoster's server IP

I assume that this is the reason, why www.uplawski.eu works alright. FTP and Mail point at the hoster's servers, which is without impact on the issue.

2) Host *, Type A, 46.38.243.234
The registrar's IP

I am afraid this is still a (client-)software issue, as the other browsers cope with the situation.
----- Edit End --------

Quote:

Does the web server properly return the www name with every request?

Tell me how to verify this.

Quote:

Is the www site a virual web site or is it the primary site on that host?

Primary.

Quote:

What I suspect is that the QuteBrowser is reporting the host address requested and does not switch to the host address replied by the server, possibly caused by not actually switching to a different physical host address.

If there is a configuration option to set, I guess I'll have to register with github again... or drop QuteBrowser quite simply.

boughtonp · 03-09-2022, 10:29 AM

All this speculation can be avoided/answered by getting QuteBrowser to display/log HTTP requests and responses, then comparing the request that has the issue to the one that doesn't.

computersavvy · 03-09-2022, 12:29 PM

The log files usually are at /var/log/httpd/ on the server which usually shows all communications.
How to get the browser to log its info is another story.

teckk · 03-09-2022, 01:41 PM

I've not used it, I see that it's python3, pyqt5, Qt5webengine

https://github.com/qutebrowser/qutebrowser

Quote:

All this speculation can be avoided/answered by getting QuteBrowser to display/log HTTP requests and responses

https://github.com/qutebrowser/quteb...qutebrowser.py

Code:

valid_flags = ['debug-exit', 'pdb-postmortem', 'no-sql-history',
               'no-scroll-filtering', 'log-requests', 'log-cookies',
               'log-scroll-pos', 'log-sensitive-keys', 'stack', 'chromium',
               'wait-renderer-process', 'avoid-chromium-init', 'werror',
               'test-notification-service']

Qt5webengine isn't having any problems with https://uplawski.eu

Code:

./insp https://uplawski.eu/

Logfile is: insp.log

GET
ResourceTypeMainFrame, (Top level page)
https://uplawski.eu/
______________________________________________________________________
GET
ResourceTypeMainFrame, (Top level page)
https://www.uplawski.eu/
______________________________________________________________________
GET
ResourceTypeScript, (External script)
https://www.uplawski.eu/util.js
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://www.uplawski.eu/images/english_sm.webp
______________________________________________________________________
GET
ResourceTypeStylesheet, (CSS stylesheet)
https://www.uplawski.eu/ups.css
______________________________________________________________________
GET
ResourceTypeStylesheet, (CSS stylesheet)
https://www.uplawski.eu/fonts.css
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://www.uplawski.eu/images/francais_sm.webp
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://www.uplawski.eu/images/deutsch_sm.webp
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://www.uplawski.eu/images/falling_foul.svg
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://www.uplawski.eu/images/Schreibfeder.svg
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://www.uplawski.eu/images/lautre.webp
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://www.uplawski.eu/images/backgrounds/bocageBG.webp
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://www.uplawski.eu/images/backgrounds/bocageBG_light.webp
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://www.uplawski.eu/images/olive.png
______________________________________________________________________
GET
ResourceTypeFontResource, (Font)
https://www.uplawski.eu/fonts/LinBiolinum/LinBiolinum_Re.ttf
______________________________________________________________________
GET
ResourceTypeFontResource, (Font)
https://www.uplawski.eu/fonts/LinBiolinum/LinBiolinum_Re.ttf
______________________________________________________________________
GET
ResourceTypeFontResource, (Font)
https://www.uplawski.eu/fonts/Fertigo_PRO.otf
______________________________________________________________________
GET
ResourceTypeFontResource, (Font)
https://www.uplawski.eu/fonts/Fertigo_PRO.otf
______________________________________________________________________
GET
ResourceTypeFavicon, (Favicon)
https://www.uplawski.eu/favicon.ico

I've got a python3, PyQt5, Qt5webengine inspector script, that should run like Qutebrowser, let me run it against that url.

Code:

./insppy
Enter/Paste url to Inspect: https://uplawski.eu/

https://uplawski.eu/

https://www.uplawski.eu/

https://www.uplawski.eu/util.js

https://www.uplawski.eu/images/english_sm.webp

https://www.uplawski.eu/ups.css

https://www.uplawski.eu/fonts.css

https://www.uplawski.eu/images/francais_sm.webp

https://www.uplawski.eu/images/deutsch_sm.webp

https://www.uplawski.eu/images/falling_foul.svg

https://www.uplawski.eu/images/Schreibfeder.svg

https://www.uplawski.eu/images/lautre.webp

https://www.uplawski.eu/images/backgrounds/bocageBG.webp

https://www.uplawski.eu/images/backgrounds/bocageBG_light.webp

https://www.uplawski.eu/images/olive.png

https://www.uplawski.eu/fonts/LinBiolinum/LinBiolinum_Re.ttf

https://www.uplawski.eu/fonts/LinBiolinum/LinBiolinum_Re.ttf

https://www.uplawski.eu/fonts/Fertigo_PRO.otf

Logging to: insp.html insp.req

https://www.uplawski.eu/fonts/Fertigo_PRO.otf

https://www.uplawski.eu/favicon.ico

No trouble here.

teckk · 03-09-2022, 01:52 PM

Uh, wait a minute. You have python3, PyQt5, Qt5Webkit installed already. Let me cut it down to size. You can test what you are getting with webengine.

Here:

Code:

#!/usr/bin/python

#Print web page requests to terminal and file.
#Needs Qt5-WebEngine and PyQt5

import sys
from PyQt5.QtCore import QUrl
from PyQt5.QtWidgets import QApplication
from PyQt5.QtWebEngineCore import QWebEngineUrlRequestInterceptor
from PyQt5.QtWebEngineWidgets import (QWebEngineView, 
                            QWebEnginePage, QWebEngineProfile)
#Get web page url                            
URL = 'https://uplawski.eu'

#Multiple User Agents for requests
a = ('Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_1 like Mac OS X) '
        'AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 '
            'Mobile/14A403 Safari/602.1')
            
b = ('Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:97.0) '
        'Gecko/20100101 Firefox/97.0')
#Select            
agent = b

#Intercept requests, print to terminal and file
class UrlRequestInterceptor(QWebEngineUrlRequestInterceptor):
    def interceptRequest(self, info):
        req = info.requestUrl()
        req2str = req.toString()
        with open('inspt.log', 'a') as f:
            print('%s\n' % req2str)
            f.write('%s\n\n' % req2str)

class WebEnginePage(QWebEnginePage):
    def acceptRequest(self):
        return QWebEnginePage.acceptRequest(self)

#Make a browser window, load page
if __name__ == "__main__":
    app = QApplication(sys.argv)
    
    interceptor = UrlRequestInterceptor()
    profile = QWebEngineProfile()
    profile.setHttpUserAgent(agent)
    profile.setRequestInterceptor(interceptor)
    
    browser = QWebEngineView()
    page = WebEnginePage(profile, browser)
    page.setUrl(QUrl(URL))
    page.setZoomFactor(1.2) #Zoom
    
    browser.setPage(page)
    browser.setMinimumSize(1000,800) #Browser size
    browser.show()
    sys.exit(app.exec_())

Michael Uplawski · 03-09-2022, 03:23 PM

Quote:

Originally Posted by teckk

Uh, wait a minute. You have python3, PyQt5, Qt5Webkit installed already. Let me cut it down to size. You can test what you are getting with webengine.

This works as I would have wanted. The redirection is honored. The Request-headers appear to ask for https://www.uplawski.eu, when I just use https://uplawski.eu on whatsmyip.org:

Quote:

Host www.uplawski.eu
User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) QtWebEngine/5.15.2 Chrome/87.0.4280.144 Safari/537.36
Accept */*
Cache-Control no-cache
Connection keep-alive
Accept-Encoding gzip,deflate

With firefox, the request looks alike:

Quote:

Host www.uplawski.eu
User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0
Accept */*
Cache-Control no-cache
Connection keep-alive
Accept-Encoding gzip,deflate

I have removed the current config.py for QuteBrowser but see no change. In view of the many alternatives, I tend to just give up on QuteBrowser or – at least – to not care any more. Anyway, my comprehension of your hints diminishes with each contribution.

ondoho · 03-10-2022, 01:19 AM

Which distro are you using it on? Which version of qutebrowser?
Have you also tried removing the complete ~/.config/qutebrowser directory?

I just tested v2.4.0, it redirects OK.

Michael Uplawski · 03-10-2022, 02:47 PM

Quote:

Originally Posted by ondoho

Which distro are you using it on? Which version of qutebrowser?

Debian Bookworm

qutebrowser v2.4.0

Backend: QtWebEngine 5.15.8, based on Chromium 87.0.4280.144
Qt: 5.15.2

Quote:

Have you also tried removing the complete ~/.config/qutebrowser directory?

I did (prior starting qutebrowser) and nothing changed, same behavior with https://uplawski.eu which leads to the startpage of l'AutreNet (www.lautre.net).

boughtonp · 03-10-2022, 05:02 PM

Quote:

Originally Posted by Michael Uplawski

Backend: QtWebEngine 5.15.8, based on Chromium 87.0.4280.144

So does it have F12 developer tools or not?

If so, press F12, goto the network tab, ensure both "Preserve log" and "Disable cache" options are ticked, ensure "All" is selected, clear any existing rows, then visit https://uplawski.eu/ and look at what comes back.

Post the raw HTTP request and response (i.e. use "view source" option in Chromium) for the first response.

Michael Uplawski · 03-11-2022, 03:17 AM

F12 did not do anything, :devtools changed the cursor into a gray dot (however) but scrolling was impossible. Then I noticed a small strange border on the right of the browser. Okay, I did what you told me.

Since

pen https://uplawski.eu the sought page is displayed immediately. All expected ressources are delivered, html, scripts and images, from www.uplawski.eu. F12 still does nothing but toggling the devtools closes the window. After that and in the same tab, I can navigate from and to https://uplawski.eu, without observing any strange behavior.

In any new tab, however, the “old” phenomenon returns, where instead of my start-page, that of www.lautre.net appears.

Edit some hours later: boughtonp, you have solved the problem. Call :devtools, check checkboxes, visit URL. Ω

boughtonp · 03-11-2022, 07:09 AM

So it was a caching issue then?

Michael Uplawski · 03-11-2022, 11:12 PM

Quote:

Originally Posted by boughtonp

So it was a caching issue then?

It must have been. All: Allow me to ignore forever, which cache was concerned, as I do not have the brain to think of yet another one.

Apart from that, the domains and those “sub-domains” concerned exist since I joined L'AutreNet. There were server errors which had been corrected, but *never* had there been a reason to identify their start-page with mine. Recently, there had been none such event. Let us say, this was a caching issue. Then it is nice to have been pointed at it, but to get stuff out of ... – I had emptied/deleted the QuteBrowser-cache, but anyway – ... out of the cache (whichever), you have to store it there, first.

It would not be polite to write that I do not want to know. And stuff is piling up, there. Thank you for wasting your time with me.

The biggest achievement of this thread is that I won the courage to draw that border-line into the window to finally take a look at the devtools widget.