PrivacyTests.org

Unfortunately, privacy on the web in 2024 is still not the norm! Most people in the world are still using browsers that aren't protecting their privacy. These leaky browsers are knowingly facilitating mass surveillance and making our society, our democracies, vulnerable.
What can we do about this problem? How can we ensure that someday soon, everyone is using a privacy-protecting browser? Have any creative ideas? Let's discuss!

2024-07-23 23:49:31

Google announced yesterday that they no longer intend to deprecate third-party cookies in Chrome: https://privacysandbox.com/news/privacy-sandbox-update/
In my opinion, this decision is a disaster for online privacy. Third-party cookies are a key method that advertisers and other tracking companies use to gather your web browsing history without your knowledge.

2024-07-16 22:01:07

Issue 89 of https://privacytests.org is out today!

2024-07-16 22:01:07

Issue 89 of https://privacytests.org is out today!

2024-06-25 22:49:18

Issue 88 of https://PrivacyTests.org is out today!

2024-05-25 17:07:43

- reply

@npub14yx…qtt2 Hm, actually that is a surprising result. I will look into what's happening there.

2024-05-25 00:13:39

Issue 87 of https://PrivacyTests.org is out today!

2024-05-25 00:13:39

Issue 87 of https://PrivacyTests.org is out today!

2024-05-10 17:22:27

- reply

Congratulations to the teams who deployed these new protections! Let's hope that Safari brings their new privacy protections to normal windows as well, that Samsung fixes the remaining State Partitioning leaks, and that DuckDuckGo finds a way to deploy State Partitioning protections on Android.
We'll keep monitoring browsers for their privacy improvements! ✅❌

2024-05-10 17:21:56

- reply

3. For the first time, Samsung Internet Browser for Android is now passing nearly all State Partitioning tests. This means it's much harder for a web tracker to "tag" the browser to follow you around the web. (Now the only remaining tested browser on any platform that is still failing most State Partitioning tests is
DuckDuckGo browser for Android, although it is blocking third-party cookies, which is a good start!)

2024-05-10 17:21:38

- reply

2. Safari Private Windows on Desktop are now also preventing the leak of unencrypted DNS queries. The latest test results show that when you visit a website in a Safari Private Window, the name of the website isn't leaked when the browser looks up the website's IP address.

2024-05-10 17:21:23

- reply

1. Safari Private Windows on Desktop are now protecting against leaks of users' screen size and window position, which can otherwise be used to fingerprint you as you browse the web with a number of other browsers:
https://files.mastodon.social/media_attachments/files/112/417/942/538/904/979/original/a9689301f7332135.png

2024-05-10 17:20:47

Browser privacy update
In the latest version of http://PrivacyTests.org (Issue 86), we are seeing impressive progress in privacy protections in Safari Private Windows and from Samsung Internet Browser.

2024-05-02 21:05:55

Issue 86 of https://privacytests.org is out today!

2024-04-12 19:15:36

Issue 85 of https://PrivacyTests.org is out today!

2024-04-02 23:45:31

- reply

@npub137g…6d5m Great point. Thanks!

2024-04-02 22:52:49

- reply

@npub10cg…yy0m Thanks! Good suggestions.

2024-04-02 21:52:23

- reply

@npub10cg…yy0m I haven't. Where did you see 630?

2024-04-02 21:51:43

- reply

@npub1t7k…twk2 There is room for improvement! 😀

2024-04-02 21:36:35

- reply

@npub10cg…yy0m Fair enough! But given that other feeds have posts with previews I should figure out if I'm doing it correctly.

2024-04-02 21:23:00

- reply

Trying to figure out why I'm not getting a preview image in the above post.

2024-04-02 21:20:56

This is just a test. 😉 https://PrivacyTests.org

2024-04-01 22:43:33

Issue 84 of https://PrivacyTests.org is out today!

2024-03-21 15:52:23

- reply

@npub1gv8…uu0q @npub16kx…4nrw Hi Jon -- I'm sorry you feel that way. I am committed to accuracy and objectivity and I want the data to be helpful to browser development companies.

2024-03-16 05:30:23

- reply

@npub1tk4…vrcl Thank you Scott! I always appreciate your kind words. :)

2024-03-15 23:59:19

Issue 83 of https://PrivacyTests.org is out today!

2024-03-12 20:15:04

- reply

@npub1s0c…n336 Thanks for the good question. Unfortunately, the Vivaldi browser has the tracker blocker disabled by default, so if a user fails to choose an option, or doesn't understand the options then trackers are not blocked. Vivaldi could enable the blocker by default and then it would pass most tracking content tests.

2024-03-01 20:23:31

Issue 82 of PrivacyTests.org is out today!

2024-02-19 12:13:31

- reply

@npub1335…s0kf @npub1xat…lff0 Hi -- there is a test for ECH on https://privacytests.org as well. Please see the "Misc tests" section.

2024-02-19 07:24:22

- reply

@npub1zx2…npx5 https://mastodon.social/@privacytests/111955464847715377
Yes, all browsers are tested with default settings.

2024-02-19 05:39:41

- reply

@npub1zca…fc3a @npub1xat…lff0 The usability-privacy tradeoff can be challenging, but in practice most browsers currently are nowhere close to the "Pareto front". Instead they under-protect users for the current level of usability.

2024-02-19 05:38:13

- reply

@npub1zca…fc3a @npub1xat…lff0 That's a great point and an important example, actually. The cross-session tests in https://privacytests.org are checking for exactly that privacy protection. The approach taken by Mullvad and Tor Browser (forgetting everything by default) indeed is probably too strong for many users, but browsers could pass those tests by simply clearing stored data for sites visited by the user with no significant interaction.

2024-02-19 04:09:58

- reply

@npub1zca…fc3a @npub1xat…lff0 When I worked for another browser company in the past, I repeatedly pushed to enable existing privacy protections by default. Management responded with delays and denial *precisely* because they were worried about the financial impact of reduced ability to track the large majority of users who stick with defaults.

2024-02-19 04:03:06

- reply

@npub1zca…fc3a @npub1xat…lff0 To me, it's really puzzling that Vivaldi does not enable their privacy protections by default. Everyone knows that mass surveillance is going on. So why not enable mass protection, particularly for the most vulnerable users who tend to rely on defaults?

2024-02-19 04:01:46

- reply

@npub1zca…fc3a @npub1xat…lff0 It's much better for users to rely on browser developers to carefully select the right default settings or make the needed code changes to protect their privacy. That simplifies the problem for users. Rather than having to understand web privacy in detail, the question is reduced to: which browser(s) should I use, and implicitly, which browser development group(s) should I rely on?

2024-02-19 03:58:04

- reply

@npub1zca…fc3a @npub1a3r…9clu @npub1n0m…z5ld @npub1xat…lff0 Thanks for your comments. I understand your point that in principle you could harden Firefox so that it works approximately the way LibreWolf does.
In practice, though, that's asking a lot of users. In fact it's beyond the large majority of most users to do this correctly. Changing individual privacy settings that look good can easily make things worse, not better.

2024-02-19 01:36:13

- reply

@npub1ptg…4hdh @npub1zca…fc3a @npub1a3r…9clu @npub1n0m…z5ld @npub1xat…lff0
You make a great point. It would indeed be an excellent solution if operating systems provided DNS privacy.

2024-02-19 01:33:02

- reply

@npub1zca…fc3a @npub1a3r…9clu @npub1n0m…z5ld @npub1xat…lff0 ❤️ No problem at all! I appreciate your response.

2024-02-19 01:07:02

- reply

@npub1lzu…6awy @npub1xat…lff0 As you can see on the table, in the case of Firefox, the browser enables DoH by default specifically for users who are in certain countries (US and Russia, plus Canada and Ukraine, not shown).
When the browser starts up, Firefox connects to Mozilla servers where, most likely, your IP address is used to determine your location.

2024-02-19 00:48:19

- reply

@npub1zca…fc3a @npub1a3r…9clu @npub1n0m…z5ld @npub1xat…lff0
In the case of Mullvad and Tor -- those browsers do much more than "toggle" the DoH settings. Mullvad runs its own DoH service and its browsers uses that by default (https://mullvad.net/en/help/dns-over-https-and-dns-over-tls). Tor Browser, of course, connects by default over the Tor network, which is a highly sophisticated anonymization service. It's not using DoH, but rather the Tor network prevents local eavesdroppers from seeing the browser's DNS connections.

2024-02-19 00:45:27

- reply

@npub1zca…fc3a @npub1a3r…9clu @npub1n0m…z5ld @npub1xat…lff0
I test all browsers by default; that's because users tend to stick to defaults. All browsers have settings that can potentially improve their privacy, but I don't think it's right to have privacy protections available only to power users who know how to tweak settings and which settings to tweak.

2024-02-19 00:43:29

- reply

@npub1zca…fc3a @npub1a3r…9clu @npub1n0m…z5ld @npub1xat…lff0
The tests are open source so that anyone can check that they are giving objective results. I have absolutely no desire to promote any browser; I am committed to impartial, objective test results. Allowing bias into the project would be a sad waste of the hundreds of hours I have spent building the project.

2024-02-19 00:40:29

- reply

@npub1zca…fc3a @npub1a3r…9clu @npub1n0m…z5ld @npub1xat…lff0
Hi! I run PrivacyTests independently and I created much of the site before I was employed by Brave. I work for Brave as a browser engineer, because I have devoted the past 10 years of my career to improving the privacy of various browsers (Tor and Firefox previously). My goal is for all browsers to have privacy, not just Brave, so I'm very happy when any browser improves its privacy.

2024-02-16 06:22:11

- reply

@npub1dex…a3xd @npub1uf7…x355 Results for DDG mobile browsers are on the Android and iOS pages. There's also a desktop browser, but it's currently in Beta so the results are in https://privacytests.org/nightly only for now.

2024-02-16 06:21:12

Issue 81 of PrivacyTests.org is out today!

2024-02-09 02:54:50

- reply

@npub1gv8…uu0q @npub1w74…j3vn @npub1jgp…72h8 In the meantime there's an excellent paper on telemetry and privacy in browsers by Douglas Leith here: https://ieeexplore.ieee.org/abstract/document/9374407
Unfortunately it doesn't include tests results for Vivaldi but I think the general approach is a good one.

2024-02-09 02:53:07

- reply

@npub1w74…j3vn @npub1gv8…uu0q @npub1jgp…72h8 Hi Dominic -- the tests are open source and you can run them a couple of different ways:
1. Clone https://github.com/privacytests/privacytests.org and follow the instructions in the README
2. A subset of tests run at https://privacytests.org/me.html

2024-02-09 02:51:05

- reply

@npub1gv8…uu0q @npub1w74…j3vn @npub1jgp…72h8
Sorry, I just saw this post and wanted to respond. Tracking in the browser is a problem, of course. The reason it's not mentioned in PrivacyTests is simply that I have limited personal time to work on the website, but I do plan on including information on tracking by browsers in the future. There's nothing nefarious here -- I'm not trying to hide anything about Brave or any other browser.

2024-02-06 01:54:11

- reply

@npub173v…erxg Thank you!

2024-02-06 01:45:19

- reply

@npub1tk4…vrcl Good question for @npub100c…gkwp!

2024-02-06 01:21:33

- reply

In my view, our collective ambition should be that all web browsers use encrypted DNS all the time. The Web is used by billions of people, so DNS encryption should be enabled by default for everyone. I'll continue to regularly run these tests and report the results, to encourage browsers to do the right thing. ✅❌

2024-02-06 01:21:25

- reply

We can see the situation is far from perfect, but there is reason for hope! It's clear from these results that all major browser engines (Chromium, Gecko, WebKit) have the capability to use encrypted DNS, but they need to actually turn encryption on by default. It's primarily a question of deciding to do it, rather than needing to wait for any new technology.

2024-02-06 01:21:00

- reply

Similar to Tor Browser, Brave's Private Windows with Tor keep your DNS requests private by sending them over the Tor network. And Safari is also keeping your DNS requests private in its Private Windows.

2024-02-06 01:20:49

- reply

I'm also testing desktop browsers' private windows. Here are the results:
https://files.mastodon.social/media_attachments/files/111/881/908/787/633/060/original/4bf7536a33cb7391.png

2024-02-06 01:20:04

- reply

(2) Firefox has enabled DoH by default in the United States and Russia (as well as Canada and Ukraine, not shown.)
(3) The champions here are the twin browsers from
@npub1a3r…9clu
and
@mullvadnet
. Tor Browser entirely avoids leaking DNS requests by sending them over the Tor network. Mullvad Browser, instead, enables DoH by default, using Mullvad's own DoH resolver.

2024-02-06 01:19:54

- reply

There are a few interesting things to note:
(1) Most Chromium browsers (except Opera and Ungoogled Chromium) are following Chrome's lead and upgrading unencrypted DNS to DoH for specific DNS providers (namely, Cloudflare and Google). But otherwise, these browsers are continuing to use unencrypted DNS. Safari takes a similar approach.

2024-02-06 01:19:37

- reply

Here are the test results for Desktop browsers! An ❌ indicates that the DNS request is being leaked unencrypted; a ✅ indicates that the DNS leak is prevented by encryption:
https://files.mastodon.social/media_attachments/files/111/881/904/124/588/735/original/9349e4deccfb9723.png

2024-02-06 01:19:18

- reply

The testing code runs this experiment repeatedly while varying the default DNS resolver assigned to the operating system. (Some browsers will notice when the operating system's DNS resolver offers DoH and automatically use DoH in that case.) The test also mimics taking browsers to different locations. (At least one browser turns on DoH for specific countries but not for others.)

2024-02-06 01:19:07

- reply

So: we have the technology. We can protect your DNS privacy. Which web browsers are actually doing that?
To find out, I created an open-source automated test that mimics DNS eavesdropping. Each browser is told to visit a (fake) website with a unique name it has never seen before, such as http://awbkquhcp.privacytests3.org. Then a common tool, tcpdump, listens for DNS requests that are being leaked by the browser unencrypted.

2024-02-06 01:18:55

- reply

DoH is clearly a privacy improvement. While you do need to trust the DNS resolver, at least your browser isn't leaking the names of the websites you visit to any other parties listening on the line.
https://files.mastodon.social/media_attachments/files/111/881/901/321/530/652/original/3d0733d2a374d0b0.png

2024-02-06 01:18:34

- reply

In recent years, some new, more private versions of DNS have been introduced. One of these variants is called DNS over HTTPS or DoH: in this protocol, your computer and the DNS resolver communicate over an encrypted connection. With DoH, your browser *whispers* the website name to the DNS resolver, which *whispers* the IP address back.

2024-02-06 01:18:21

- reply

Here's the weakness: most DNS requests are sent unencrypted! In almost all web browsers, every time you click a link to visit a new website, your computer metaphorically *shouts* the website name so everyone nearby on the internet can hear! And the DNS resolver *shouts* the IP address back. It's embarrassing. 😳

2024-02-06 01:18:17

- reply

The first DNS resolver software (known as BIND) was created in 1984! Which is fitting, because the DNS protocol unfortunately has an intrinsic weakness that has facilitated mass surveillance ever since.

2024-02-06 01:18:00

- reply

These days, on the internet, IP addresses are how computers "phone" one another, and how your computer knows how to connect to a website you visit. So when you enter a website name in your browser's address bar (say, http://PrivacyTests.org), your computer first sends that website name in a DNS request to a DNS resolver, which responds with the IP address for that website. Then your computer has the information it needs to connect to the website via its IP address.

2024-02-06 01:17:46

- reply

DNS is akin to the "Directory Assistance" or "Directory Inquiry" service (411 in the US and Canada). Since the old days, if you wanted to phone someone, and you didn't have their phone number, and they weren't in your phone book, you could call Directory Assistance and ask them to look up the phone number for you.
https://files.mastodon.social/media_attachments/files/111/881/896/223/290/020/original/9b96f1f90316c5d2.png

2024-02-06 01:17:09

2024-02-06 01:16:46

- reply

Here's a brief explanation of the privacy problem, before I share the test results. The problem we're investigating is connected to browsers' use of a protocol called the Domain Name System, or DNS. It's the method every web browser uses to find out the IP address of any website you want to visit.

2024-02-06 01:16:31

Browser Privacy Testing Update, a thread:
Last week, in the latest issue of http://PrivacyTests.org, I introduced results from new tests that examine whether common web browsers are leaking the names of the websites you visit via the DNS protocol. The short answer is "😱" but there's hope that this can change!

2024-02-03 03:38:35

- reply

New in this issue: DNS privacy tests!

2024-02-03 03:38:16

Issue 80 of http://PrivacyTests.org is out today!

2024-01-30 17:29:55

- reply

@npub1w74…j3vn @npub1gv8…uu0q @npub1jgp…72h8 Yeah, making it more mobile-friendly is something aspire to do one day :)

2024-01-30 14:54:46

- reply

@npub1jgp…72h8 @npub1a5j…ja98 The Waterfox browser is on my radar! I've been focused on adding more tests but I am definitely considering it.

2024-01-30 14:52:55

- reply

@npub1w74…j3vn @npub1gv8…uu0q @npub1jgp…72h8 Interesting site! I had not seen it before.

2024-01-30 14:51:51

- reply

@npub1w74…j3vn @npub1gv8…uu0q @npub1jgp…72h8
Yes, it's all tested with default settings. It says "default settings" in the top-left corner of every table.
Regarding Brave browser -- I think these are good questions for Brave. I run this account and the PrivacyTests project independently. (I do plan to have testing of telemetry in the future.)

2024-01-30 06:34:38

- reply

@npub1w74…j3vn @npub1gv8…uu0q @npub14r3…jqc6
Anyway, Jon has repeatedly questioned my motives but I don't question his. Jon says he cares about privacy and I choose to believe him. So I fully expect he will be shortly turning on more privacy protections, such as the tracker blocker by default and tracking query parameter blocking for the benefit of Vivaldi's users. I will heartily and publicly celebrate those advances as soon as they happen.

2024-01-30 06:29:57

- reply

@npub1w74…j3vn @npub1gv8…uu0q @npub14r3…jqc6
Jon says I "test our browser with privacy features turned off" but those features were turned off by Vivaldi, not me. Jon could have the tracker blocker turned on by default but he chooses not to.

2024-01-30 06:29:17

- reply

@npub1w74…j3vn @npub1gv8…uu0q @npub14r3…jqc6
I put far too much effort into PrivacyTests to use it to promote one browser. I'm happiest when all browsers pass tests because it's good for human rights.

2024-01-30 06:28:45

- reply

@npub1w74…j3vn @npub1gv8…uu0q @npub14r3…jqc6
I'm not sure why Jon feels that my employment at Brave is "quite hidden" on the website. It's not something I'm hiding at all. But I didn't build PrivacyTests for Brave. I built it independently because I care about web privacy. That I care about privacy is the reason I have worked on browser privacy and Tor, Mozilla, and Brave over the past decade.

2024-01-22 20:22:50

- reply

@npub1cvc…4xmj A warning before opening the page is actually even stronger than HTTPS by Default. It's called HTTPS-Only Mode.
I'm considering Waterfox for future tests.

2024-01-22 17:08:35

- reply

Given that the large majority of web users are using browsers with HTTPS by Default, there shouldn't be any technical obstacle for Firefox and Safari to follow suit. Hoping Firefox and Safari will enable HTTPS by Default soon. I'll keep testing and let you know when that happens!
✅❌

2024-01-22 17:08:19

- reply

Next, our tests show that all Chromium-based browsers are providing HTTPS by Default. That means: whenever you visit a web page, the browser tries to use HTTPS and only falls back to HTTP if HTTPS is unavailable.
https://files.mastodon.social/media_attachments/files/111/800/699/978/464/105/original/e2bac8874e35e935.png

2024-01-22 17:07:58

- reply

Congratulations to the Safari team for making these valuable privacy advances!

2024-01-22 17:07:45

- reply

In addition, Safari Private Windows are blocking tracking query parameters and protecting users against the leak of their screen resolution and window position:
https://files.mastodon.social/media_attachments/files/111/800/697/585/149/182/original/42ef252ce8813a13.png

2024-01-22 17:07:17

Browser Privacy Update (a thread)
In the latest version of http://PrivacyTests.org , we can see progress in privacy protections: Safari 17.2 has partitioned the Blob API, joining Brave and Firefox-based browsers. Now only Chrome and some Chrome-derived browsers are still leaking data through the Blob API:
https://files.mastodon.social/media_attachments/files/111/800/695/749/759/820/original/68cc97cc3fcaad32.png

2024-01-20 03:14:06

Issue 79 of PrivacyTests.org is out today!

2024-01-06 04:35:33

Issue 78 of https://PrivacyTests.org is out today!

2023-12-23 19:28:19

- reply

I'm hoping all of the remaining cross-site leaks on all platforms will be fixed by the end of 2024. We're in the home stretch! Next year we will continue to monitor the deployment across browsers of other kinds of privacy protections, including HTTPS enforcement, fingerprinting protections, tracker blocking, and a variety of other important measures against mass surveillance! ✅❌

2023-12-23 19:28:06

- reply

The explanation for these good results is simple: all browsers on iOS must be based on WebKit, the engine developed for Safari, which has partitioning protections built in. One exception is a leak often found in the favicon cache. Because a favicon, which is a website's logo displayed in a browser tab, is part of a browser's user interface, the behavior perhaps not controlled by WebKit code and therefore needs to be separately fixed in each browser.

2023-12-23 19:27:51

- reply

Finally, on iOS we have seen excellent partitioning for a long time in all browsers:
https://files.mastodon.social/media_attachments/files/111/631/379/279/946/103/original/464de6ff5d22430d.jpeg

2023-12-23 19:27:23

- reply

Ironically, DuckDuckGo and Samsung are blocking third-party cookies but not partitioning the other leaks. I hope this oversight will be corrected soon!

2023-12-23 19:27:04

- reply

On Android browsers, we are also seeing major progress. In the last two months, Chrome, Edge, and Opera have shipped thorough State Partitioning protections. Browsers from
DuckDuckGo, Samsung, Vivaldi and Yandex are still lagging behind, however:
https://files.mastodon.social/media_attachments/files/111/631/376/134/930/102/original/64001a7475b6c153.jpeg

2023-12-23 19:26:21

- reply

The most important of these leaks is the notorious third-party cookie. A number of browsers are still not blocking or partitioning third-party cookies: Chrome, Edge, Opera, and Vivaldi. Chrome has stated they will block third-party cookies this year; the other three browsers are perhaps waiting for Chrome to make this move. They shouldn't wait! Our privacy should be protected today.

2023-12-23 19:26:14

- reply

Clearly, the past two years have been a time of major progress in browser privacy. But there remain a few crucial leaks of private data before we can declare that desktop browsers are full partitioning.

2023-12-23 19:25:20

- reply

Contrast the above results with the test results from two years ago (Issue 2, 2021-10-25), when only Tor Browser was passing the full suite of State Partitioning tests:
https://files.mastodon.social/media_attachments/files/111/631/369/280/584/642/original/5608ef009e80e225.png

2023-12-23 19:24:43

- reply

By providing comprehensive State Partitioning, a browser makes it harder for you to be tracked as you browse the web. For the first time, all desktop browsers tested are passing most of the state partitioning tests. Note the sea of green!
https://files.mastodon.social/media_attachments/files/111/631/366/923/044/411/original/81e207afd6238d00.jpeg

2023-12-23 19:24:03

Browser Privacy Progress Update
In the latest version of http://PrivacyTests.org (Issue 77), we see that Vivaldi desktop has now adopted substantial State Partitioning protections, joining all other desktop browsers in protecting users against cross-site data leaks. Congratulations to the Vivaldi team for making this important privacy feature available to their users!

2023-12-22 01:28:21

Issue 77 of https://PrivacyTests.org is out today!

2023-12-21 22:35:09

- reply

@npub15c0…hxke It's good that you have disabled the Topics API, but will you block third-party cookie tracking now or will you wait for Chrome to do it first?

2023-12-18 18:35:15

- reply

@npub14yx…qtt2 @npub15c0…hxke It would be good if Vivaldi could update their statement in light of the forthcoming partitioning changes.

2023-12-11 03:22:20

- reply

@npub173f…8z2p I think it may only work when DoH has been enabled. I am working on a more extensive test of this.

2023-12-10 06:35:50

Issue 76 of https://PrivacyTests.org is out today!