How is a custom user agent overridden?

ondoho · 02-09-2022, 11:32 AM

Quote:

Originally Posted by JASlinux

Please don't feel obligated to address issues not of interest.

What I addressed is of interest to your question, whether you like to believe it or not.

You also don't really explain where & how duckduckgo.com "knows" that you are using Brave.
I had a look - there's a button that says "Add DuckDuckGo to Firefox" (for me). Is that what you mean?

Looking at the scripts the page loads, there's also this one: https://duckduckgo.com/d3039.js
Unlike all others, it contains the strings "brave", "download" and "badge-link", which is the CSS class of the button.

Unsurprisingly, when I disable javascript that button is gone.

So, that's your answer.

Bad communication right from the start in this thread, you really need to learn to ask questions better. Please click on & read the first link in my signature, and afterwards also this: http://catb.org/~esr/faqs/smart-questions.html

JASlinux · 02-09-2022, 06:52 PM

Quote:

Originally Posted by evo2

Sounds like you messed up your browser settings. When I do what you describe above, duckduckgo.com regurgitates whatever user agent string I send it.

Behold (insights appreciated).

Actual user agent string:

Code:

Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.107 Safari/537.36

whatismybrowser.com:
https://i.postimg.cc/N0x7NFwR/whatis...-useragent.png

DuckDuckGo:
https://i.postimg.cc/bvP9Hb4z/ddg-brave-useragent.png

I am typing in Brave, but not 87.0.4280.107.

JASlinux · 02-09-2022, 06:57 PM

Quote:

Originally Posted by boughtonp

To understand how websites can get information not sent via HTTP headers, press F12 and examine the output of "console.dir(navigator)" then read the documentation of those properties.

This could provide more insight, but both normal and HTML searches report the same smart potato DuckDuckGo user agent other reporting sites miss or neglect to feedback.

JASlinux · 02-09-2022, 07:27 PM

netcat/ncat aren't in my Ubuntu-based system. I installed netcat_1.10-41 from the Ubuntu Universe repository and there's still no recognition of the commands.

Again the goal is just to understand how DuckDuckGo knows I am using Brave despite my string, and perhaps to learn if other sources have access to the same info.

Quote:

Originally Posted by teckk

You could also do that with

First terminal
netcat -l -p 8100 -v

Second terminal
curl -H'xxx' http://127.0.0.1:8100

etc

Edit:
or

netcat -l -p 8100 -v

Code:

agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0"

curl -A "$agent" -H'xxx' http://127.0.0.1:8100

teckk · 02-10-2022, 08:43 AM

Did you look at Braves source to see what it is doing?

https://brave.com/download/

https://github.com/brave/brave-browser

I can change my web browsers user agent with scripts on or off and I get the same results with duckduckgo. I set a weird user agent on purpose so you can see it. But the same user agent both times.

Scripts/images off
https://0x0.st/oXt-.png

Scripts/images on
https://0x0.st/oXtH.png

Having javascript on, letting the page scripts run did not allow duck to better probe my machine to get some "secret data". Not unless your browser is made to tell info. I suspect the Brave devs want to advertise their browser. That is common on web browsers. Also a lot of Browsers have javascript as part of their code base. If you turn scrips off, you kill part of the browser. Like webengine for example. No scripts, no web inspector.

That is why I showed you how to read the header that your browser is sending.

And for netcat. Look and see what Ubuntu is calling that executable. netcat, nc, ncat, etc.

I can tell you exactly what loads when you search duckduckgo for "what is my user agent", letting all scripts run, all images load.

This:

Code:

GET
ResourceTypeMainFrame, (Top level page)
https://duckduckgo.com/html/
______________________________________________________________________
GET
ResourceTypeMainFrame, (Top level page)
https://html.duckduckgo.com/html/
______________________________________________________________________
GET
ResourceTypeStylesheet, (CSS stylesheet)
https://duckduckgo.com/h2035.css
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://duckduckgo.com/assets/logo_homepage.normal.v108.svg
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://duckduckgo.com/assets/loupe_gray.png
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://duckduckgo.com/assets/loupe_white.png
______________________________________________________________________
GET
ResourceTypeFavicon, (Favicon)
https://duckduckgo.com/favicon.ico
______________________________________________________________________
POST
ResourceTypeMainFrame, (Top level page)
https://html.duckduckgo.com/html/
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/www.whatsmyua.info.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/whatsmyuseragent.org.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/whatmyuseragent.com.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/whatuseragent.com.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/webbrowsertools.com.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/www.whatismyuseragent.net.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/user-agents.net.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/en.wikipedia.org.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/useragent.openadmintools.com.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/useragentstring.com.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/www.deviceinfo.me.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/gs.statcounter.com.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/developer.mozilla.org.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/yandex.com.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/www.useragents.com.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/deviceatlas.com.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://external-content.duckduckgo.com/ip3/sitedozor.ru.ico
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://duckduckgo.com/t/sl_h
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://duckduckgo.com/assets/logo_header.v108.svg
______________________________________________________________________
GET
ResourceTypeImage, (Image - jpg/gif/png/etc)
https://duckduckgo.com/assets/chevron.png
______________________________________________________________________

teckk · 02-10-2022, 08:49 AM

Oh, and I just realized I was using ducks html page. Ok let me set a user agent and go to it's full page with all browser features turned on.

https://0x0.st/oXtA.png

And...I'm not using an iPhone.

It's Brave Browser that is reporting Brave Browser.

ondoho · 02-11-2022, 10:03 AM

Quote:

Originally Posted by teckk

https://0x0.st/oXtA.png

Hadn't thought of trying that as a search query.
Once again, the result is quite different with javascript on or off.

teckk · 02-11-2022, 11:42 AM

The OP could make a little tester for that to better understand it.

Brave looks like a big bunch of code to try and troubleshoot. So are a lot of other browsers.

Since Brave is gtk3 already. A tester for DuckDuckgo user agent reporting using webkit2gtk in gtk3 can be made easy enough. Any halfway recent version of webkit2gtk, gtk3, glib will do.

ducktest.c

Code:

//webkit2gtk user agent tester for duckduckgo

#include <gtk/gtk.h>
#include <webkit2/webkit2.h>

//DuckDuckGo user agent page, may change.
const gchar* DuckPage = "https://duckduckgo.com/?q=what+is+my+user+agent&t=h_&ia=answer";
                
//User agent to test here.
const gchar* agent = "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_1 "
                "like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) "
                "Version/10.0 Mobile/14A403 Safari/602.1";

static void destroyWindow(GtkWidget* widget, GtkWidget* window) {
    gtk_main_quit();
}

static gboolean closeWebView(WebKitWebView* webView, GtkWidget* window) {
    gtk_widget_destroy(window);
    return TRUE;
}

int main(int argc, char* argv[])
{
    gtk_init(&argc, &argv);
    g_set_application_name("DuckTest"); //Browser name

    GtkWidget *main_window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
    
    //Window size for test
    gtk_window_set_default_size(GTK_WINDOW(main_window), 1000, 800);

    WebKitWebView *webView = WEBKIT_WEB_VIEW(webkit_web_view_new());
    
    //Zoom level for the window
    webkit_web_view_set_zoom_level(WEBKIT_WEB_VIEW(webView), 1.5);
    
    //Browser settings. Set scripts TRUE or FALSE here for test. Add more if you want.
    WebKitSettings *settings = webkit_settings_new();
    g_object_set(G_OBJECT (settings), "enable-javascript", FALSE, NULL);
    g_object_set(G_OBJECT (settings), "auto-load-images", FALSE, NULL);
    g_object_set(G_OBJECT (settings), "minimum-font-size", 18, NULL);
    g_object_set(G_OBJECT (settings), "enable-developer-extras", TRUE, NULL);
    g_object_set(G_OBJECT (settings), "print-backgrounds", FALSE, NULL);
    g_object_set(G_OBJECT (settings), "user-agent", agent, NULL);

    //Set it from above.
    webkit_web_view_set_settings(WEBKIT_WEB_VIEW(webView), settings);

    gtk_container_add(GTK_CONTAINER(main_window), GTK_WIDGET(webView));
    
    //Connect to signals
    g_signal_connect(main_window, "destroy", G_CALLBACK(destroyWindow), NULL);
    g_signal_connect(webView, "close", G_CALLBACK(closeWebView), main_window);
    
    //Load the page
    webkit_web_view_load_uri(webView, DuckPage);
    gtk_widget_grab_focus(GTK_WIDGET(webView));
    gtk_widget_show_all(main_window);

    //Here we go
    gtk_main();

    return 0;
}

//gcc ducktest.c -o ducktest $(pkg-config --cflags --libs gtk+-3.0 glib-2.0 webkit2gtk-4.0)

dugan · 02-11-2022, 08:28 PM

Chrome and Firefox have Developer Tools, which show you the exact headers that your browser is sending for each request. I assume Brave does too. Those should help.

Now, for the record, I have no idea how you got that DuckDuckGo screenshot. It looks nothing like what I see when I type "what is my user agent" into their search box.

teckk · 02-12-2022, 08:49 AM

Quote:

Now, for the record, I have no idea how you got that DuckDuckGo screenshot. It looks nothing like what I see when I type "what is my user agent" into their search box.

Yes, I can see why that would be.

I get different looking pages with different user agents. With scripts/images on or off. Also if I am using Ducks full page or html only page. The iPhone agent pages are lighter and load fewer scripts. Which would be a reason to alter the user agent that the browser reports to start with. If one doesn't care, then leave it alone.

This is lighter, with less content. It's what you need to use with scripts off.
https://duckduckgo.com/html/

This is heavier, with more content. You'll get different looking pages depending on user agent.
https://www.duckduckgo.com/

In fact, let me see...I get a little different looking pages with each, iPhone 10, iPhone 12, Win10 Firefox, Win10 Edge, Linux Firefox. But Duckduckgo only gives me the user agent that I used, nothing else. (Answer to the OP)

Ones that I used.

Code:

Mozilla/5.0 (iPhone; CPU iPhone OS 15_0_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/78.0.257670029 Mobile/19A348 Safari/604.1

Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_1 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Mobile/14A403 Safari/602.1

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0

Mozilla/5.0 (X11; Arch; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36 Edg/97.0.1072.69