Integrating a real-time coaching tool into generative AI systems can significantly increase user awareness of algorithmic bias and help people craft more inclusive prompts, according to a latest study from researchers at Penn State and Oregon State University.
The research focuses on a specific intervention designed to disrupt the tendency of text-to-image generators to produce stereotypical or non-inclusive content. By providing immediate feedback as a user types, the tool encourages a moment of reflection, forcing a pause in the creative process to consider how a prompt might inadvertently trigger biased outputs.
The findings, which were presented on April 16 at the 2026 Association for Computing Machinery (ACM) Conference on Human Factors in Computing Systems in Barcelona, Spain, suggest that embedding media literacy directly into the software is more effective than traditional “outside-the-medium” warnings. The paper was recognized with an honorable mention from the conference’s awards committee.
As a former software engineer, I’ve seen how often “bias” is treated as a backend problem to be solved by engineers through dataset scrubbing. This research shifts the focus toward the user’s role in the loop, suggesting that a coaching tool alerts users to AI bias not just to fix the image, but to educate the human operator.
Moving Media Literacy Inside the Machine
Traditionally, media literacy efforts—such as those used to combat misinformation on social media—occur after the fact or as external warnings. Users are often told about the dangers of an algorithm before they open an app or after they have already consumed a piece of content.
This new approach integrates the intervention directly into the workflow of text-to-image generation. As users enter their descriptions, the “inclusive prompt coaching” tool identifies potential bias triggers and issues warnings, offering specific suggestions to make the prompts more inclusive before the image is ever generated.
“Oftentimes, media literacy interventions like those for social media occur outside of the medium, informing or warning users about the dangers of social media before or after they’ve interacted with it,” said study co-author S. Shyam Sundar, Evan Pugh University Professor and the James P. Jimirro Professor of Media Effects at Penn State. “Here we are using the medium itself – AI text-to-image generators – to educate users about how to better utilize the medium while they’re interacting with it. It’s a newer twist on the media literacy approach to address the problem of lack of inclusiveness in generative AI.”
The Impact on User Trust and Confidence
To test the efficacy of the coaching tool, researchers recruited 344 participants via an online survey platform. The participants were split into three distinct groups to measure how different levels of guidance affected their interaction with the AI:
- Inclusive Prompt Coaching: Users received warnings about algorithmic bias and suggestions for inclusive phrasing.
- Detailed Prompt Coaching: A control group receiving technical guidance without the specific focus on inclusivity.
- No Coaching: A control group with no interventions.
The results indicated that those using the inclusive coaching tool showed a marked increase in their awareness of algorithmic bias—the systemic tendency of AI to reproduce societal stereotypes. Beyond awareness, the intervention boosted the users’ confidence in their own ability to write prompts that would yield less biased results.
Crucially, the study noted an improvement in “trust calibration.” Here’s the ability of a user to accurately judge when a system is reliable and when it is likely to fail. By highlighting the system’s flaws in real-time, the coaching tool helped users adjust their trust levels to better reflect the actual trustworthiness of the AI.
The Trade-off: User Experience vs. Education
Despite the educational gains, the research revealed a significant friction point: the user experience (UX). The study found that the inclusive prompt coaching intervention led to a less satisfactory overall user experience. This highlights a classic tension in AI design—the conflict between “seamless” interaction and “meaningful” friction.
When a tool is too seamless, users may blindly trust the output. When a tool introduces “friction” (such as a warning or a suggestion to rewrite), it slows the user down. While this friction is exactly what enables the media literacy intervention, it can make the software feel more cumbersome or restrictive to the complete user.
Understanding the Stakeholders
The implications of this research extend across several groups within the tech ecosystem:
| Stakeholder | Primary Benefit | Primary Challenge |
|---|---|---|
| End Users | Higher AI literacy and more inclusive results. | Slower workflow and interrupted UX. |
| AI Developers | Reduced output of harmful stereotypes. | Designing friction that doesn’t alienate users. |
| Researchers | New framework for “in-medium” literacy. | Scaling interventions across different AI modalities. |
For developers, the challenge is now to find a “middle path”—creating a coaching mechanism that alerts users to bias without degrading the product’s usability to the point where users disable the feature.
The Path Forward for Generative AI
As generative AI becomes deeply embedded in professional design and communication workflows, the risk of automating stereotypes increases. The move toward “trust calibration” suggests that the goal should not be to create a “perfect” AI—which may be impossible given the biased nature of the training data—but to create a more discerning user.
The next step for this line of research will likely involve testing these interventions across other modalities, such as Large Language Models (LLMs) used for writing, to see if similar “in-medium” coaching can reduce bias in text-based outputs.
For those interested in the technical specifics of the intervention, the full research details are available through the Penn State research portal.
Do you think AI tools should interrupt your workflow to warn you about bias, or should the “fix” happen entirely behind the scenes? Share your thoughts in the comments.
