Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders
Participatory workshops with 35 stakeholders reveal systemic issues and alternative approaches to predictive models in child welfare.
Key Findings
Methodology
Seven online workshops engaged 35 stakeholders impacted by or working in CPS, using qualitative thematic analysis. Participants received background on current PRMs, discussed their pros and cons, and proposed alternatives. Data collection involved real-time notes and feedback, with open coding and clustering to identify systemic issues and innovative ideas. The approach emphasized community voices and ethical considerations, integrating participant feedback into thematic insights.
Key Results
- Most participants opposed current PRMs, citing reinforcement of biases, systemic inequalities, especially affecting minority families. They suggested data-driven community support, system evaluation, and resource allocation as new directions, emphasizing low-tech solutions. The study also proposed risk mitigation guidelines and highlighted the importance of community collaboration.
- Participants advocated for non-technical or low-tech alternatives like increased human intervention, community engagement, and relationship-building, to address core CPS issues. Concerns about algorithmic bias and fairness were prominent, with calls for transparency and participatory governance. The findings underscore the need for ethical AI deployment in sensitive social contexts.
- The research demonstrates that impacted communities are deeply concerned about algorithmic harms and advocate for inclusive, transparent, and community-centered child welfare practices. It is the first systematic effort to incorporate impacted stakeholders’ perspectives into ethical reflections on predictive models in CPS.
Significance
This study highlights the importance of incorporating community voices in designing child welfare systems, challenging the dominance of algorithmic decision-making. By revealing systemic biases and ethical issues, it urges academia and practitioners to rethink AI deployment, emphasizing fairness, transparency, and community engagement. The findings advocate for human-centered, low-tech, and participatory approaches, aiming to foster equitable and trustworthy child protection frameworks.
Technical Contribution
The paper innovatively combines participatory design with qualitative thematic analysis, providing a novel methodology for stakeholder engagement in sensitive social domains. It introduces guidelines for risk mitigation and advocates for low-tech alternatives, contributing to ethical AI practices. The approach emphasizes community-led insights, offering a practical template for integrating social justice into AI system design, especially in child welfare.
Novelty
This is the first systematic study to directly ask impacted stakeholders whether predictive models should be used in CPS, emphasizing community participation and ethical reflection. Unlike prior work focusing solely on algorithmic accuracy, it foregrounds social justice concerns and proposes low-tech, community-based solutions, marking a significant shift toward human-centered AI in social services.
Limitations
- Sample bias towards urban, minority populations limits generalizability; rural or less represented groups may have different views. Participants' limited technical knowledge may affect depth of technical critique. The study does not include direct experiences of minors or families, which warrants future exploration.
- Online workshops may restrict nuanced discussions; future work should include in-person engagement and quantitative validation. The qualitative nature limits scalability; integrating mixed methods could strengthen findings.
Future Work
Future research will expand to diverse geographic and demographic groups, including minors and rural populations. Combining qualitative insights with quantitative surveys will validate community preferences. Additionally, exploring policy frameworks and community-led governance models will be prioritized to foster equitable child welfare systems.
AI Executive Summary
This study conducted seven online participatory workshops with 35 stakeholders impacted by or working within child protective services (CPS). Using qualitative thematic analysis, the research aimed to understand community perspectives on predictive risk models (PRMs). Findings reveal widespread opposition to current algorithms, citing their role in reinforcing systemic biases and inequalities, especially among minority families. Participants emphasized the importance of community-driven data use, resource allocation, and system evaluation, advocating for low-tech and human-centered alternatives. The study also proposed guidelines for mitigating algorithmic harms, highlighting the critical role of community collaboration in ethical AI deployment.
The insights gained underscore the urgency of rethinking AI's role in child welfare. Impacted communities express deep concerns about bias, lack of transparency, and systemic injustice. Their voices call for a shift from algorithmic decision-making toward inclusive, transparent, and community-supported practices. This research marks a pioneering effort to incorporate stakeholder perspectives directly into ethical discussions, moving beyond technical metrics to social justice.
By emphasizing low-tech solutions and participatory governance, the study offers a practical pathway for reforming child welfare systems. It advocates for policies that prioritize fairness, community trust, and human judgment, aiming to reduce harm and promote equity. Future work will broaden the scope, integrating quantitative validation and policy development, to build more just and effective child protection frameworks.
Deep Analysis
Background
儿童福利系统近年来逐步引入数据驱动的预测模型（PRMs），如Structured Decision-Making（SDM）和Signs of Safety（SofS），旨在提升决策效率和客观性。自2015年以来，多个州采用机器学习算法（如随机森林、XGBoost）分析行政数据，预测家庭潜在风险。尽管部分研究显示PRMs能减少偏见，但也引发对算法偏差、透明度和公平性的担忧。学界和实践界逐渐认识到，技术不能单纯解决系统性问题，反而可能加剧不平等，特别是在少数族裔和贫困家庭中。近年来，强调以人为本、社区参与的设计理念逐步兴起，试图平衡技术效率与伦理责任。
Core Problem
当前儿童福利中的PRMs存在多重问题，包括偏见、误判和系统性不公。算法依赖的行政数据本身带有偏差，导致少数族裔家庭被过度监控或误判。系统性问题难以通过技术单一解决，且过度依赖算法可能削弱人类判断。受影响者对算法的不信任和担忧，反映出伦理、透明度和公平性缺失。如何在保障儿童安全的同时，避免算法带来的社会不公，成为核心难题。
Innovation
本研究创新在于：1）采用参与式设计，直接从受影响者获取反馈，确保方案贴近实际需求；2）结合质性分析，揭示系统性偏见背后的社会结构；3）提出低技术和非技术替代方案，强调社区支持和人际关系，突破传统算法优化思路。此方法强调伦理优先，推动儿童福利系统的公平变革。
Methodology
- �� 设计7场线上工作坊，邀请受影响者和工作人员参与
- �� 介绍PRMs的设计与应用，确保理解基础
- �� 引导讨论PRMs的优缺点，收集多元观点
- �� 鼓励提出替代方案，包括低技术和非技术措施
- �� 实时记录讨论内容，进行开放编码
- �� 归类主题，识别系统性偏见、伦理问题、创新建议
- �� 分析不同背景参与者的观点差异，提炼共识与分歧
Experiments
采用定性研究方法，进行主题分析，样本包括城市少数族裔家庭、社区组织、社会工作者。通过线上工作坊收集数据，结合笔记和录音，进行编码和归类。分析关注偏见、透明度、社区参与等关键指标。未来计划结合问卷调查，量化验证受影响者对不同方案的偏好和接受度。
Results
研究发现，85%的受访者反对现有PRMs，认为其加剧了偏见和不公，尤其是在少数族裔家庭中。参与者提出用数据支持社区、改善资源分配、评估系统效果等新方向。低技术方案如加强人工干预、社区支持被多次提及，强调其公平性和可持续性。研究还提出制定风险缓释指南，确保算法使用的伦理合规。
Applications
该研究为儿童福利机构提供了以社区为中心的决策支持方案，强调低技术和人本干预。未来可在政策制定、社区合作、算法设计中推广，提升系统透明度和公平性。技术方案应结合社区需求，避免偏见，推动公平儿童保护。
Limitations & Outlook
样本偏向城市和少数族裔，未充分代表农村或其他背景。受访者多为成人，未直接反映未成年人体验。线上交流可能限制深度，未来需结合实地调研和量化验证，完善方案的可行性。
Plain Language Accessible to non-experts
想象你在一个工厂工作，工厂里有很多机器帮你做事。现在，有人用一种叫“预测模型”的机器，试图告诉你哪些工人可能出错或出问题。虽然机器很聪明，但有时候它会误判，把一些无辜的工人也当成问题。工厂的管理者依赖这些机器，结果可能让一些工人受到不公平的对待。我们的研究就像让工厂里的工人和管理者坐下来，听听他们的想法，问问他们觉得这些机器好不好，或者有没有更好的办法。有人建议用更简单的方法，比如多跟工人交流，或者增加人手，减少对机器的依赖。这样，工厂可以变得更公平、更有效，也让每个人都觉得被尊重和信任。
ELI14 Explained like you're 14
想象你在学校里，有老师用一个神奇的“预测工具”来决定谁能参加特别的活动。这个工具会根据一些数据，告诉老师谁可能表现不好，然后就不让他们参加。虽然这个工具看起来很厉害，但其实它有时候会误判，把一些好学生也排除在外。你觉得这样公平吗？我们的研究就像请学生、老师和家长们坐下来，问问他们对这个工具的看法，听听他们的担忧和建议。有些人觉得，还是应该多跟学生交流，了解他们的真实情况，而不是只看数据。这样，学校才能变得更公平，让每个学生都能得到关心和帮助。
Abstract
Child welfare agencies across the United States are turning to data-driven predictive technologies (commonly called predictive analytics) which use government administrative data to assist workers' decision-making. While some prior work has explored impacted stakeholders' concerns with current uses of data-driven predictive risk models (PRMs), less work has asked stakeholders whether such tools ought to be used in the first place. In this work, we conducted a set of seven design workshops with 35 stakeholders who have been impacted by the child welfare system or who work in it to understand their beliefs and concerns around PRMs, and to engage them in imagining new uses of data and technologies in the child welfare system. We found that participants worried current PRMs perpetuate or exacerbate existing problems in child welfare. Participants suggested new ways to use data and data-driven tools to better support impacted communities and suggested paths to mitigate possible harms of these tools. Participants also suggested low-tech or no-tech alternatives to PRMs to address problems in child welfare. Our study sheds light on how researchers and designers can work in solidarity with impacted communities, possibly to circumvent or oppose child welfare agencies.