AI Has A Darker Side In Healthcare, Expert Says

TL;DR

For listenersOffer from Amazon

Turn quiet afternoons into listening time

• Thousands of audiobooks, podcasts and originals
• Listen on your phone, tablet or Echo — also offline
• Cancel anytime

Try Audible free →Free trial for new members

As an affiliate, we earn on qualifying purchases.Hide

At the Association of Health Care Journalists Fall Summit in Los Angeles, UC Berkeley researcher Mishal Khan warned that AI in healthcare can harm workers as well as help them. Concerns include loss of clinical skills, increased monitoring and algorithmic control, weakened patient relationships, and unclear accountability when AI contributes to an error.

Researchers warned at the Association of Health Care Journalists Fall Summit in Los Angeles that artificial intelligence in healthcare could bring harms for workers, including erosion of clinical skills, closer monitoring and reduced control over work. Mishal Khan, a senior researcher at the UC Berkeley Labor Center, said that AI’s effects should not be measured only by whether it eliminates jobs: workers could face worsening conditions even if they keep their positions.

Khan grouped workplace AI uses into electronic monitoring, algorithmic management and automation of particular tasks. Monitoring can involve cameras or other systems tracking workers. Algorithmic management may use software to analyze worker-customer conversations and offer prompts about how employees should respond. Automation refers to assigning specific parts of a job to AI.

In Khan’s account, the risks include job loss, work product being taken, de-skilling, lower wages or wage theft, faster work and injuries, discrimination, limits on labor organizing, and loss of autonomy. She said de-skilling was a particular concern in healthcare because reliance on automation could affect workers’ skills and scope of practice.

Stanford biomedical informatics professor Tina Hernandez-Boussard described medical residents who were sent to a hospital cardiac ward in Uganda. When the facility lost electricity and AI was unavailable to read electrocardiograms, the residents struggled to synthesize information confidently enough to make recommendations, she said. Her account was presented as an example of how dependence on automated tools could affect clinicians’ readiness when systems are unavailable.

How Automation Could Change Care Work

The concerns reach beyond employment numbers. If clinicians become less practiced at tasks handled by software, they may have less confidence when technology is unavailable or fails. That could affect clinical decision-making, although the account from Uganda does not establish how common this problem is or show that AI caused a particular patient outcome.

Khan also raised questions about patient relationships and accountability. Automated handoffs could leave workers feeling detached from patients, she said, while an error involving an AI recommendation may make it difficult to determine who is responsible. Those issues matter to patients as well as staff because they touch communication, trust and the handling of mistakes.

Workplace systems may also influence staffing and pay. Khan warned that algorithmically managed nursing platforms could have nurses bid for shifts and use predictions about the lowest wage a nurse might accept. That was a warning about a possible model, not a report that such practices are widespread across healthcare.

Monitoring, Management and Automation

Khan placed healthcare AI within a longer history of workplace technology. She said digital tools have been used for decades to increase the extraction of labor and worsen working conditions. Her argument was that AI can extend existing workplace practices, rather than creating an entirely new set of concerns.

One area she highlighted was ambient documentation tools, sometimes called ambient scribes. Although these systems are presented as tools for patient safety, Khan said workers want to know what information is collected about them and whether it could later be used to make employment decisions. The source report did not identify a specific system or describe evidence that such data has been used against workers.

Khan also pointed to policy efforts. She said laws passed in Illinois and Vermont prohibit replacing mental health workers with AI, and that a California law bars companies from advertising an AI chatbot as a health professional. A separate California bill under consideration would prevent workers from being required to train their replacements, including an AI replacement. These examples indicate that lawmakers are addressing some concerns, but do not resolve the broader workplace questions.

“Even if they don’t lose a single job to AI, it doesn’t mean there are no harms to workers.”

— Mishal Khan, senior researcher at the UC Berkeley Labor Center

Evidence and Accountability Questions

The summit report describes expert concerns and examples, but it does not quantify how often healthcare workers experience these harms or establish that AI has caused widespread job loss, lower pay, injuries or reduced clinical competence. The residents’ experience in Uganda was offered as an illustration; details about the hospital, the residents’ preparation and patient outcomes were not provided.

It also remains unclear what data ambient scribes collect about workers, how employers may use those records, and who would be legally responsible if an AI-supported recommendation contributes to a mistake. The report does not specify the status or implementation details of the laws and proposed bill beyond Khan’s account.

Policy and Workplace Decisions Ahead

The next developments will depend on how healthcare employers deploy AI, what safeguards they provide for worker data and skills, and whether lawmakers advance additional protections. The California bill described by Khan was still under consideration at the time of the report; no later legislative outcome is provided.

For now, the speakers’ remarks call attention to questions health systems will need to address as they adopt these tools: whether workers retain the ability to perform core tasks, how automated decisions are reviewed, how staff and patients are affected, and who answers for errors. The summit report does not set out a specific implementation plan or timetable.

Key Questions

What did the experts warn about?

They described possible workplace harms, including de-skilling, monitoring, reduced autonomy, lower wages and weakened patient relationships. These were concerns raised by the speakers, not findings that each harm is occurring throughout healthcare.

Does the report say AI is already replacing healthcare workers?

No. Khan discussed job loss as one possible risk but stressed that workers can be harmed even if they do not lose their jobs. The report does not provide figures on healthcare jobs replaced by AI.

What is de-skilling in this discussion?

It refers to workers losing practice in specialized tasks when automation takes over those tasks. Khan said this could affect healthcare workers’ skills and scope of practice.

What laws or proposals did Khan mention?

Khan said Illinois and Vermont had passed laws prohibiting replacement of mental health workers with AI, and that a California law bars marketing an AI chatbot as a health professional. She also described a California bill under consideration that would address requiring workers to train their replacements, including AI systems.

Source: rss