LQ needs to review the security of your connection frustrating delays

linux-man · 05-09-2023, 07:43 PM

@Whoever owns LQ or manages it

Can we get an update as to Admin's solution to this thread's issue? What's been done so far and what is going to be done can we get an update from forum owners?

Who owns this forum? Is this a for-profit forum?

Jan K. · 05-10-2023, 02:46 AM

Quote:

Originally Posted by linux-man

... You mean you never encounter Ray ID Cloud Fare checks most of the time?

Not even once...

Perhaps your issue is Tor related?

hazel · 05-10-2023, 10:26 AM

Quote:

Originally Posted by Jan K.

Perhaps your issue is Tor related?

I found this interesting blog about Cloudflare and Tor which I think may be relevant to the problems tor users have been seeing, despite being quite old.

wpeckham · 05-10-2023, 10:45 AM

Quote:

Originally Posted by linux-man

@Whoever owns LQ or manages it

Can we get an update as to Admin's solution to this thread's issue? What's been done so far and what is going to be done can we get an update from forum owners?

Who owns this forum? Is this a for-profit forum?

Look back up the thread for comments from Jeremy. That is the closest thing to what you are asking for, and he IS the authority here.

He does not owe us exact code or settings, and were I him I would be seriously reluctant to disclose those details. That said he has always been responsive, informative, and very reasonable. (MUCH more than I would be, I can be rather a jerk when pressed.)

In this case he adjusted his side to try to avoid the issues mentioned. Obviously he cannot change Cloudshare or your ISP (Or Onion) but what he can do to help he has done. Now is a time for testing and reporting any change in the behavior.

martixy · 05-14-2023, 07:48 PM

Getting a lot of those myself. Like every 5 min or so.

GentleThotSeaMonkey · 05-15-2023, 01:53 AM

Might any of this (or similar) be ...?
https://www.zenrows.com/blog/bypass-cloudflare

hazel · 05-15-2023, 05:46 AM

That's a scraper site! It's because of these scraper bots stealing data (I assume for training AIs) that organisations like clownflare exist to plague honest users like us. They will find ways around all barriers. We are the ones who end up suffering from their effects.

Jan K. · 05-15-2023, 09:47 AM

Support LQ and it's choice of host by simply using an "approved" browser...

hazel · 05-15-2023, 09:51 AM

Quote:

Originally Posted by Jan K.

Support LQ and it's choice of host by simply using an "approved" browser...

OK, I use firefox. That's approved, isn't it? I've never used tor. But I still get served with capchas and other rubbish when I try to help some poor soul with a scripting problem.

pan64 · 05-15-2023, 11:44 AM

Quote:

Originally Posted by hazel

OK, I use firefox. That's approved, isn't it? I've never used tor. But I still get served with capchas and other rubbish when I try to help some poor soul with a scripting problem.

Don't worry, be happy. Nothing is perfect (but me).

Jan K. · 05-16-2023, 06:31 AM

Quote:

Originally Posted by hazel

OK, I use firefox. That's approved, isn't it? I've never used tor. But I still get served with capchas and other rubbish when I try to help some poor soul with a scripting problem.

Okay, so much for my blind shootings in the dark!

Not a fan of Mozilla, so only have chrome (on android tablet) and Vivaldi on pc (Windows/Slackware)... at least using these to access LQ I've never been bothered with logging or captcha issues.

That's quite a narrow selection, isn't it?

Perhaps we could get a list of "approved" browsers? Or better, what exactly is causing the problems?

teckk · 05-16-2023, 09:38 AM

Post #51 tells about it. That has been my experience. I knew that cloudflare was fingerprinting, and did not like a fake user agent. It likes even less a user agent reporting an operating system different than what one is using. But, not always. It depends on what is between the code tags.

I knew that python with urllib/requests needs to report a browser like request header to access a cloudflare page.

Post #51 explains what is happening quite well. I don't mean to use their product, but what cloudflare is doing and why.

As far as firefox on a linux machine, without a user agent change, without beeing on a tor node, with scripts turned on, no script or add blockers, and it still kicks you out...

I can usually post to LQ, in a web browser with images, scripts, plugins, DnsPrefetch, java, XSS Auditing Enabling, cross site cookies, turned off. And pretending to be an iPhone.

Code:

int main(int argc, char **argv)
{
    QCoreApplication::setOrganizationName("xxxx");
    QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
    QCoreApplication::setAttribute(Qt::AA_UseHighDpiPixmaps);

    QApplication app(argc, argv);
    app.setWindowIcon(QIcon(QStringLiteral(":AppLogoColor.png")));

    //Browser settings
    QWebEngineSettings::defaultSettings()->setAttribute(QWebEngineSettings::PluginsEnabled, false);
    QWebEngineSettings::defaultSettings()->setAttribute(QWebEngineSettings::JavascriptEnabled, false);
    QWebEngineSettings::defaultSettings()->setAttribute(QWebEngineSettings::AutoLoadImages, false);
    QWebEngineSettings::defaultSettings()->setAttribute(QWebEngineSettings::LocalStorageEnabled, false);
    QWebEngineSettings::defaultSettings()->setAttribute(QWebEngineSettings::DnsPrefetchEnabled, false);

Like that, but not always.

And that was with this user agent.

Code:

const char* agent = "Mozilla/5.0 (iPhone; CPU iPhone OS 16_2 "
"like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) "
"CriOS/113.0.5359.112 Mobile/15E148 Safari/604.1";

pan64 · 05-16-2023, 09:55 AM

Quote:

Originally Posted by teckk

Post #51 tells about it. That has been my experience. I knew that cloudflare was fingerprinting, and did not like a fake user agent. It likes even less a user agent reporting an operating system different than what one is using.

Hm. What kind of security is it? that means faking user agent and os is pointless. Or the browser is not secure enough?

teckk · 05-16-2023, 10:33 AM

Faking use agent has always been pointless for those web sites that fingerprint you. It can get you a different web page delivered to you from servers that deliver different web page formats depending on what user agent you send.

Different lighter page. Less busy, fewer scripts, lower bandwidth, faster loading. That's the only reason that I do it. Someone like Hazel that has limited bandwidth could benefit from that.

Also, if you are wanting the page source to parse, different agent will get you different source code.

Turning javascript off is better than a particular user agent. But not all web sites will work without it. LQ will.

teckk · 05-16-2023, 11:31 AM

Quote:

that means faking user agent and os is pointless

Since anyone can say anything, I'll take screengrabs.

A little webkit2gtk 2.40.1 browser with images - javascipt on, With a stock webkit2gtk user agent. Browser opened to http://127.0.0.1:8100

Code:

ncat -klp 8100 -v
Ncat: Version 7.93 ( https://nmap.org/ncat )
Ncat: Listening on :::8100
Ncat: Listening on 0.0.0.0:8100
Ncat: Connection from 127.0.0.1.
Ncat: Connection from 127.0.0.1:57288.
GET / HTTP/1.1
Host: 127.0.0.1:8100
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4 Safari/605.1.15
Accept-Encoding: gzip, deflate
Accept-Language: en-US
Connection: Keep-Alive

Open iftop in a terminal, Clear out the browsers cache, and open newegg.com.
https://0x0.st/HN32.png

Ok, this time a qt5-webengine 5.15.13 browser with images - javascipt on. With a mobile user agent. Browser opened to http://127.0.0.1:8100

Code:

ncat -klp 8100 -v
Ncat: Version 7.93 ( https://nmap.org/ncat )
Ncat: Listening on :::8100
Ncat: Listening on 0.0.0.0:8100
Ncat: Connection from 127.0.0.1.
Ncat: Connection from 127.0.0.1:43388.
GET / HTTP/1.1
Host: 127.0.0.1:8100
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 16_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/113.0.5359.112 Mobile/15E148 Safari/604.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Accept-Encoding: gzip, deflate, br

Open up iftop in terminal again. Clear the browsers cache, then load newegg.
https://0x0.st/HN3_.png

10.6MB to load the opening page with the desktop agent, 4.4MB with the mobile agent.
And it's lighter all the way through. What you can't do is navigate all the way to purchase with it. As soon as newegg starts checking you, when you get to the payment page, you are kicked out. You'll need to report a real linux/chrome user agent to make it through payment.

You are good until "they" start examining you. I use it for bandwidth saving and a more enjoyable experience browsing the web.

Quote:

Or the browser is not secure enough?

Different web browser engines get a different response, using the same user agent. Haven't tried them all but, webkit, webengine, gecko, plaemoon, get different responses from the same web site, with the same fake user agent. You can see from the posts above, they report different headers. And you can manipulate that too. The different browser engines definitely interact with a web server differently.

You say more or less secure. They are using different ssl, tls, the way they use session cookies...long subject, and I'm an amateur.