Ever seen HTTP error 403 for old user agent values?
Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Ever seen HTTP error 403 for old user agent values?
Since many years ago, I eventually use wget with a custom user agent option - with a nonsense value or not.
Today, for a fairly complex start page of a not too big bussiness, I got a 403 error! Strange... I changed the user agent to a "better" one, more close to current browsers versions, and it worked. I downloaded index.html (for the "/" I asked) that way.
Terminal output showing two wget commands describing the last paragraph:
Code:
$ wget1 'https://[domain]/' # old user agent
--2018-09-20 10:02:56-- https://[domain]/
Resolvendo [domain] ([domain])... [ipv6a], [ipv6b], [ipv4a], ...
Conectando-se a [domain] ([domain])|[ipv6a]|:443... conectado.
A requisição HTTP foi enviada, aguardando resposta... 403 Forbidden
2018-09-20 10:02:56 ERRO 403: Forbidden.
$ wget2 'https://[domain]/' # fairly common user agent
--2018-09-20 10:03:28-- https://[domain]/
Resolvendo [domain] ([domain])... [ipv6a], [ipv62b], [ipv4b], ...
Conectando-se a [domain] ([domain])|[ipv6a]|:443... conectado.
A requisição HTTP foi enviada, aguardando resposta... 200 OK
Tamanho: não especificada [text/html]
Salvando em: “index.html”
index.html [ <=> ] 423,87K 710KB/s in 0,6s
2018-09-20 10:03:30 (710 KB/s) - “index.html” salvo [434047]
$ alias|grep wget
alias wget='wget --no-check-certificate'
alias wget2='wget -U "************************************"'
alias wget1='wget -U "******************************"'
$
So, with the old UA sending, am I using something that people trying to remotely execute things do? This is bad, very bad, because that is far from everything I ever do. The most "complicated" thing I usually try is to save things that annoying websites show, but use "fancy" tricks to block them being saved.
Which user agent are you mimicking? It could well be that the site in question just refuses connections from old user agents that it no longer supports.
The old user agent is 'Internet Explorer 6'. I just wanted the same HTML... I think that websites that sent different pages for different browsers have serious basic problems, and I usually do not like them.
One of the most knowledgeable people I know will display a page that says "I won't show you anything in an IE browser" on his website. It's the main reason I discovered Firefox.
It's interesting that the site you asked about is just throwing an error instead of telling you why. That, I agree, is rather rude. [but is is their site; their rules -- so it goes]
The old user agent is 'Internet Explorer 6'. I just wanted the same HTML... I think that websites that sent different pages for different browsers have serious basic problems, and I usually do not like them.
The problem is that websites sometimes have to send different HTML to different browsers due to incompatibilities in how they interpret HTML and CSS, for example, a problem that was much more marked in the past. Internet Explorer 6 is extremely old and was in fact considered to be one of the most problematic of all browsers when it came to not interpreting the standards correctly. On top of that, hardly anyone uses Internet Explorer 6 any more.
So, the site in question cannot produce the same HTML for all the browsers out there. That's the reason that it has obviously decided not to send HTML at all to some old deprecated standard-incompatible browsers for which they would have to modify their code.
One of the most knowledgeable people I know will display a page that says "I won't show you anything in an IE browser" on his website. It's the main reason I discovered Firefox.
hahahaha... >,<
And must that be updated to talk about Edge too? Or Edge is acceptable?
About the rest you wrote. It is their site. It did not bug me much, except that it wants to make me click on dozens of images to see them in their full size. A script to parse all the images, transform them in their big sister image was (luckily, could be harder) fairly easy to do).
One of the most knowledgeable people I know will display a page that says "I won't show you anything in an IE browser" on his website. It's the main reason I discovered Firefox.
It's interesting that the site you asked about is just throwing an error instead of telling you why. That, I agree, is rather rude. [but is is their site; their rules -- so it goes]
Perhaps, but having had to code websites to handle Internet Explorer 6 and chums in the past, I can understand their actions.
hahahaha... >,<
And must that be updated to talk about Edge too? Or Edge is acceptable?
About the rest you wrote. It is their site. It did not bug me much, except that it wants to make me click on dozens of images to see them in their full size. A script to parse all the images, transform them in their big sister image was (luckily, could be harder) fairly easy to do).
Perhaps, but having had to code websites to handle Internet Explorer 6 and chums in the past, I can understand their actions. :)
There are many pages that could be reduced, with *all* their important things being showed only with HTML 3.2. A huger number of them can be expressed in HTML 4.01 + CSS 1 or 2 (if I remember correctly), including aspects for mobiles.
HTML 5 and the DRM things I hear and heard going around are not something I like very much. I like old browsers, and systems that work with them. They are just very fine, and there is no need to change either for the rest of our lives.
There are many pages that could be reduced, with *all* their important things being showed only with HTML 3.2. A huger number of them can be expressed in HTML 4.01 + CSS 1 or 2 (if I remember correctly), including aspects for mobiles.
HTML 5 and the DRM things I hear and heard going around are not something I like very much. I like old browsers, and systems that work with them. They are just very fine, and there is no need to change either for the rest of our lives.
With all due respect, dedec0, you've obviously never constructed websites, not to any great extent anyway and not over the last few decades. Html5 is excellent progress on previous incarnations of HTML, as is CSS3 an important step forward from previous incarnations of CSS. Or would you prefer, to take just one example, positioning things on the screen by using tables which were never designed for that purpose?
But, I can see that I won't be able to persuade you otherwise, so I won't bother posting links to pages that discuss why web design is much easier, and much more powerful, than it was several years ago.
In saying that, we agree on DRM, and I admit that, having started coding in the 1980s, I am in favour of information-rich sites as they generally used to be rather than the sound-bite sites with huge buttons and lots of blank space that are in vogue these days - but that is a separate issue from the technology that is being used to build the sites.
My contact with HTML, CSS and webdesign in general were limited, indeed. But I have many complaints in many websites today... and most of my complaints (at least) can be solved with a fairly small bit of HTML 4 and possibly a bit of CSS too. And a fair amount of them could be arranged (fairly easily) to be HTML3 only, if needed! HTML4+CSS already condemned the "tables to format" thing, so this is not something new in HTML5 - as I guess and think you know; I found this part of your #13 post strange to read.
The pointers to discussions will be welcome. But I would prefer that they are easy to read, going step by step to add details and to deepen ideas, and that they do not make it hard for people with less specific or deep backgrounds in the area (like me) to understand and (possibly) learn something.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
Ever seen HTTP error 403 for old user agent values?
