Authors:
(1) Yuwen Lu, he contributed equally to this work, University of Notre Dame, USA;
(2) Chao Zhang, he contributed equally to this work and work done as a visiting researcher at the University of Notre Dame;
(3) Yuewen Yang, Work done as a visiting researcher at the University of Notre Dame;
(4) Yaxin Yao, Virginia Tech, USA;
(5) Toby Jia-Jun Li, University of Notre Dame, USA.
Table of Links
3 CO-DESIGN WORKSHOPS
In the first phase of our study, we conducted 5 in-person exploratory co-design workshops[6] to achieve the following goals:
(1) Exploring users’ perceived disruptiveness and annoyance of different dark patterns in various usage contexts;
(2) Learning the existing measures that users have developed or adopted, consciously or subconsciously, to cope with dark patterns;
(3) Investigating users’ needs and expectations regarding dark pattern intervention techniques.
3.1 Participants
We recruited 12 participants (PA1–PA12; 5 men, 7 women) through word of mouth, email mailing list, and flyer distribution. Our participants represent diverse backgrounds in occupational domains (e.g., health, education, social assistance, and information services), Internet usage (ranging from 2–5 hours to 8+ hours per day), and knowledge of dark patterns (7 had heard of the concept, the remaining 5 had not). Detailed demographics can be found in Appendix A.1. We conducted 5 in-person workshops with 2–4 participants in each session. The groups were divided based on participants’ time availability. Each participant was compensated $30 for their time.
3.2 Workshop Activities
Each workshop lasted 2 hours and started with a brief introduction to the concept and examples of dark patterns. The participants completed three activities together: a focus group discussion, a storyboard fill-in session, and a tangible website redesign activity.
3.2.1 Focus Group Discussion. In a focus group, participants were first introduced to the concept of dark patterns, then reflected on and shared dark patterns examples they previously encountered in everyday lives. During the discussion, researchers provided feedback and clarifications to help them understand the boundary and varied “darkness” level of dark patterns [38]. The participants were then asked to rank their examples by the level of perceived annoyance and disruptiveness and provide explanations. Researchers followed up with questions to find out the current strategies adopted by participants to address the impacts of dark patterns.
We designed this activity to help participants ground their understanding of dark patterns’ prevalence and impact in their concrete personal experiences. Reflecting on dark patterns and the associated level of annoyance also acted as a stimulus, prompting participants to contemplate countermeasures in subsequent activities. The format was intentionally less structured, with an emphasis on encouraging speak-up and fostering a comfortable environment that would promote open dialogue in subsequent activities.
3.2.2 Storyboard Fill-in. The second activity was a storyboard fill-in. Storyboards are commonly used HCI tools to visually communicate user experience scenarios to the audience [119, 122, 134]. In co-design workshops, storyboards help contextualize participants and prompt them to think about their needs, goals, and constraints in the scenarios described [9, 90]. In our study, we adopted “fill-in-the-blanks” storyboards [88, 129] to explore the users’ desired solutions to dark patterns. For each storyboard, we left one or two frames blank, encouraging participants to provide insights into their understanding of dark patterns and their preferred abilities to counter these patterns [7].
Specifically, we presented participants with 6 storyboards (Fig. 1) depicting scenarios in which a tool helped them mitigate the negative impacts of dark patterns. Of these, 3 focused on improving users’ awareness of dark patterns (Fig. 1a), and 3 (Fig. 1b) others on helping users take act against dark patterns. Each storyboard contained 4 frames, representing the background of the scenario, the tool used, how the tool helped, and the desired outcome respectively. We left the second or/and the third frames blank and asked participants to brainstorm their desired tools and their interactions.
3.2.3 Tangible Website Redesign. The last activity was a tangible user interface redesign. While storyboard fill-in focused on the design of intervention scenarios (e.g. the user flow of interventions), this activity targeted intervention at a lower, more granular level—the specific alternatives of dark patterns on UI design. We designed this activity to be tangible, in the form of paper prototyping instead of digital UI mockup modification, to encourage participants to make bold changes and think outside the box [113]. It also avoids the learning process for a new digital design tool.
We curated 13 representative dark pattern examples on 7 websites for online shopping, flight booking, video streaming, and social media as a diverse set of scenarios. These examples were selected from previous research literature [11, 38], online discussions of dark patterns [15], and the researchers’ own experiences, with the goal of triggering discussions among participants. In each workshop, participants selected around 5 dark patterns that they were most concerned about to work on. For each dark pattern instance, we provided a printout of the website interface and a set of cut-out UI widgets from the same interface. During the activity, participants edited the cut-out widgets and drew new UI widgets using a variety of provided stationery[8], and re-assembled them into a more desired design of the original website interface. The interfaces created by the participants were collected for analysis [9].
3.3 Workshop Findings
Following the open coding methods [14, 63], two researchers conducted a thematic analysis of audio transcriptions for workshops and materials produced in the three activities. The researchers conducted three rounds of iterative labeling, in which descriptive labels on relevant transcript pieces were created, grouped, and generalized into higher-level themes. The analysis was discussion-based, with no necessity for inter-rater reliability due to the aim of discovering emergent themes [84].
Comparative analysis was not the focus of the study, but a comparison of responses from tech industry users and other users showed no significant difference in their perceptions or coping mechanisms concerning dark patterns. This is in line with previous work [55] which showed no significant differences between end users and experts in perceptions of dark patterns.
Our workshop findings (WF) are described below in response to our workshop goals.
WF1: Users would like to learn more about the impact of dark patterns. Participants expressed their desire to know more about the potential impacts of dark patterns. Although many were able to detect dark patterns, most participants only developed a vague assumption about the impacts, falling short of articulating the specifics. The particular mechanisms and impact remain as a “blackbox”, confirming findings in [11, 35].
Participants often asked about the detailed mechanisms and impact of dark patterns and felt the knowledge was useful. Importantly, clear knowledge of dark patterns’ impact can help users choose services more consciously and potentially reduce the irritation of seeing dark patterns. PA6 and PA7 mentioned that for disguised ads on Instagram, “those are annoying at first, but once you know that (its impact) and come to expect it, it’s like okay (less annoying) (PA6)”. These findings complement existing research by demonstrating users’ autonomy—they are not merely passive consumers of dark patterns. They are interested in actively learning, and the acquired knowledge can change their usage behavior and connections with online platforms.
WF2: Users’ perceptions of dark patterns are personal and dynamically changing, which are formed based on user preferences, types of dark pattern instances, and usage contexts. Despite the prevalent negative perception of manipulative UX design patterns, not all participants viewed them unfavorably. In fact, responses varied widely from negative to positive, echoing findings from studies on online behavioral advertising [124]. Users often perceived a persuasive pattern as helpful when it aligns user goals with stakeholder profits. PA2 liked the autoplay feature on Netflix, even when understanding it used forced continuation, because “it is useful when I am away from my mouse”. Similarly, for disguised ads on social media, PA12 expressed that “I won’t block them. I usually don’t engage or buy stuff... Maybe I’ll see something in the future I like”. We summarized 3 most common perceptions from participants: disruptiveness (the user experience was disrupted by the dark pattern), indifference (the dark pattern was neither harmful nor useful), and helpfulness (the pattern was helpful in the current context).
Users encountering a dark pattern typically evaluate its potential pros and cons subconsciously, influenced by factors like perceived convenience, potential consequences, and the pattern’s apparent malicious intent. Accordingly, three factors—the user, the dark pattern, and the usage context— determine this perception. An example is the autoplay feature for the next episode on Netflix. PA1, PA10, and PA12 expressed their dislike of this feature, while PA2, PA5, and PA11 generally thought it was convenient. Furthermore, even the same user’s perception can shift with different usage contexts and changing goals. PA11 found auto-play harmful when using Netflix during work because of the short break time, but useful when casually browsing after work just for fun. These findings are in parallel with results in [35] and enhance previous findings by highlighting the highly individualized and contextualized nature of users’ perceptions of dark patterns.
WF3: Users develop varied coping mechanisms based on their different dark pattern perceptions. This finding reveals more details regarding how end users react when facing dark patterns’ perceived influences, in addition to the conclusions in [11]. For disruptive dark patterns, many users actively seek solutions to mitigate the impact. PA12 used a calendar to track the end of free subscription trials as a reminder to unsubscribe. On Instagram, PA6 developed a habit to avoid disguised ads when tapping through all stories. “Funnily enough, every time I watch a story, I have developed... an unconscious habit, I close out (by swiping down) and I click the next one. (PA6)” If the user can successfully find a solution, it becomes a “muscle memory” for them. For the Instagram habit, PA6 expressed that “I didn’t know why I do that, but I guess that’s a dark pattern and I am unconsciously adapted.” PA1 also developed the habit of reaching their mouse before an episode ends on Netflix, to wiggle the cursor in time and avoid forced continuation.
For dark patterns that users feel indifferent to, the most common strategy is ignoring them. PA6 mentioned that they gradually got used to “confirmshaming” dark patterns and “just don’t care anymore”. When asked about a disguised advertisement on a flight booking website during the redesign activity, PA8, PA9, and PA10 reported that they did not even notice it. They considered it “too colorful” to be relevant and therefore simply ignored it.
3.4 Design Implications
The thematic analysis results of our co-design workshops offered several design implications for dark pattern intervention techniques and user empowerment.
DI1: Empower users with the ability to make changes on dark patterns. When encountering disruptive dark patterns, users often feel manipulated but have no ability to resist (WF1). To help participants regain self-autonomy [82], we can empower end users with the ability to change the interfaces of dark patterns. It would help users take the initiative to mitigate the negative impact.
DI2: Provide information on the potential consequences of dark patterns. During our workshops, participants expressed the need for information on the potential influences of dark patterns and envisioned a ranking of their severity. With such information, users can make better informed evaluations of the impact of a dark pattern on themselves (WF2) and develop their coping mechanism accordingly (WF3).
DI3: Offer users multiple intervention options for each dark pattern. Perceptions of dark patterns may shift with users, types of dark pattern instances, and usage contexts. Even for the same dark pattern, users may act differently (WF3). As a result, it is necessary to have multiple intervention options for users to choose from. In this way, users can have more flexibility in personalization and autonomy.
DI4: Design dark pattern interventions with three strategies: interface design change, user flow adjustment, and behavioral outcome reflection. In our redesign activity, participants proposed intervention techniques for dark patterns that can be categorized into three approaches: (1) modifying interface components and layouts to eliminate malicious design, (2) adjusting user flows to prevent users from falling into behavioral traps, and (3) evoking reflection by uncovering the outcomes of dark patterns for long-term self-change. Future intervention designs can take inspiration from these strategies and apply them in appropriate scenarios.
For example, on a flight booking website, while the website highlighted the more expensive first-class and main-cabin options over the basic economy, PA8, PA9, and PA10 changed them to the same size and color to pursue a fair style. PA1, PA2, and PA3 designed an agent to provide an appropriate action guide with dark patterns on Amazon to help them save money. PA6 and PA7 wanted to know, in the long term, how many times dark patterns on a certain website affected their behavior, to reflect on their relationship with the platform.
We used these findings and design implications from our co-design workshop to guide our second co-design phase—a technology probe study.
[6] The protocol of workshops has been reviewed and approved by the IRB at our institution.
[7] Examples of participant-filled storyboards are available in the supplemental materials.
[8] The provided stationery included but was not limited to pencils, colored pens, scissors, highlighters, and glue sticks
[9] Examples of participant-redesigned interfaces are available in the supplemental materials.
This paper is available on arxiv under CC 4.0 license.