Euro S&P 2023: DarkDialogs – Automated detection of 10 dark patterns on cookie dialogs

This blog post is based on the paper “DarkDialogs: Automated detection of 10 dark patterns on cookie dialogs” [1], presented during the 8th IEEE European Symposium on Security and Privacy in Delft, Netherlands.

Delft, Netherlands

Introduction

Cookie dialogs (see Figure 1) are commonly seen on websites to protect user privacy. They seek permission from users before processing personal data, offering privacy-conscious individuals the option to refuse consent and safeguard their privacy. Gathering consent benefits websites and their advertising partners by ensuring legal compliance and potentially boosting sales.

Figure 1: Example of a cookie dialog (taken from https://www.eurosport.com/). There is no easy way to refuse the cookies. That is one of the examples of a “dark pattern”.

In reality, websites often design interfaces to make it harder to opt-out, utilizing tactics known as dark patterns. These manipulative strategies range from omitting opt-out buttons (as shown in Figure 1) or disregarding user decisions to employing subtle techniques like convoluted language or visually emphasizing the opt-in button through color coding.

Dark patterns undermine user autonomy and violate privacy laws. In 2022, because of non-compliant dialogs, Google and Meta were fined by the French Data Protection Authority (CNIL) [2].

In the paper “DarkDialogs: Automated detection of 10 dark patterns on cookie dialogs“, authors aimed to build an automatic tool to search for dark patterns and evaluate how widespread those dark patterns are. The paper claims that their tool extracts cookie dialogs with 98.7% accuracy and detects different dark patterns (in total, they have 10 of them) with 99% accuracy.

In their work, Selenium web scraping library for Python was used to make the interaction with websites automatic. Also, ChromeDriver was used to load and interact with websites using the Chrome browser.

Methodology

At a high level, their methodology of finding can be summarized in 4 steps (also see Figure 2):

Figure 2: Analysis flow of a single web-site
  1. Cookie dialog detection: find potential cookie dialogs on the page, rank them using a range of factors (see Figure 3) and select the dialog with the highest ranking.
  2. Clickable locator: collect and classify all the clickable elements on the selected dialog. An example of clickable elements on a cookie dialog is shown in Figure 4.
  3. Cookie behavior analysis: try to click on different elements (ex. accept all, reject all), collect cookies, and assess how clicking different elements impacts the total number of installed cookies.
  4. Dark pattern detection: automatically detect dark patterns using the predefined rules.
Figure 3: Cookie dialog ranking system
Figure 4: Examples of clickable elements

Dark pattern examples

In this section, a couple of examples of dark patterns are given.

Highlighted accept button

Figure 5: Agree button is highlighted more.

Ambiguous close

Figure 6: Close button with ambiguous (does it accept or reject cookies?) function is present.

Multiple dialogs

Figure 7: Multiple dialogs. What happens if contradicting options are chosen?

Dataset

The 2021 Tranco list of most popular websites was used as the main dataset. From it, the following 3 smaller datasets were constructed:

  1. Top500: 500 most popular web sites. This dataset was human-labelled to validate an accuracy of presented solution.
  2. Rand500: randomly selected 500 web-sites chosen from top million web-sites. This dataset was also human-labelled to validate the accuracy of presented solution.
  3. Rand10000: randomly selected 10k web-sites chosen from top million web-sites. This dataset was made to analyze a broad view of cookie dialogs.

Results

Cookie dialog detection

Figure 8: Data collection results for all datasets with information about (i) how many websites were loaded; (ii) how many of them had a cookie dialog; (iii) how many of them had a disagreement between a human and automatic evaluation.

Clickable element distribution on the cookie dialogs

Figure 9: Percentage of the websites having the corresponding clickable element.

Dark pattern detection

Among the web-sites from Rand10k dataset, 3744 dark patterns were detected in total.

Figure 10: Prevalence percentage of each dark pattern on the dialogs

Conclusion

In the paper “DarkDialogs: Automated detection of 10 dark patterns on cookie dialogs” authors presented an automated system to detect dark patterns on cookie dialogs with 99% accuracy. They analyzed 10k websites and found that most use at least one dark pattern.

Their code is open-source and available on GitHub.

References

[1] DarkDialogs: Automated detection of 10 dark patterns on cookie dialogs. / Kirkman, Daniel; Vaniea, Kami E; Woods, Daniel W. 8th IEEE European Symposium on Security and Privacy. IEEE, 2023.

[2] EDPB Annual Report 2022. https://edpb.europa.eu/system/files/2023-04/edpb_annual_report_2022_en.pdf

SHARE
TwitterFacebookLinkedIn