# Curbside Chatbots: Auditing the Mental Health Interventions Large Language Models Recommend to Clinicians, Caregivers, and Patients

Source: [arXiv](https://arxiv.org/abs/2611.12814v1)  
Feed7 permalink: https://feed7.dev/p/2611-12814v1-017n6ph  
Published: 2026-11-09T18:19:30.000Z  
Trust: Needs Review (needs_review)

## Why Included

An audit found that mental health recommendations from language models varied with whether a clinician, caregiver, or patient described the case. It is a reminder to test advice across user contexts.

## Source Summary

Researchers tested **41 language models** on **19 case vignettes**, collecting **35,055 responses** across clinician, caregiver, and self-described scenarios. They found that recommendation patterns differed by who presented the case.

## Practical Implication

For systems that answer health questions, evaluate responses across user roles and check whether named interventions match the model’s own ratings. The audit found that models sometimes named options they rated below zero as initial choices when asked about a specific intervention type.

## Agent-Ready Context

Researchers tested **41 language models** on **19 case vignettes**, collecting **35,055 responses** across clinician, caregiver, and self-described scenarios. They found that recommendation patterns differed by who presented the case.

For systems that answer health questions, evaluate responses across user roles and check whether named interventions match the model’s own ratings. The audit found that models sometimes named options they rated below zero as initial choices when asked about a specific intervention type.

This is an audit of recommendations, not evidence about clinical outcomes. The instrument, code, and corpus are planned for release on publication.

## Connected Context

Feed7 judgment across 1045 accumulated Signals:

This Signal adds user role as a distinct axis for auditing clinical recommendations: the same case can elicit different advice for clinicians, caregivers, and patients, and a model may recommend an intervention it rates negatively. It supports checking both role-specific responses and consistency between ratings and recommendations, while leaving clinical outcomes untested.

- [MedPRESS: A Multi-turn Benchmark for Patient-Pressure-Induced Medical Sycophancy in LLMs](https://feed7.dev/p/2608-02520v1-16negmk) — Both show that clinical guidance can shift with how a request is framed; MedPRESS varies user pressure over multiple turns, while this audit varies the user’s role and checks recommendation-rating consistency.
- [Patient-specific rubric-based evaluation of large language models across longitudinal cancer care](https://feed7.dev/p/2610-35847v1-0se27hb) — Both caution that aggregate clinical performance can hide differences across contexts: NPC-RubricBench examines care phases, while this audit examines who presents the case.

## Context Map

- Layer: benchmark
- Domains: research
- Topics: None

## Uncertainty

- This is an audit of recommendations, not evidence about clinical outcomes. The instrument, code, and corpus are planned for release on publication.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.
