AI, healthcare and organisational change

The moderator asked the panel to be candid about the problems that don't make the press, so these notes keep to the general lessons rather than specific hospital stories.

The model is the easy part

• Understand the process deeply. Walking the unit once isn't enough: sit with the people who know the system, because the same variable can mean different things to clinicians and to data teams.
• Trust between clinical teams and data teams is essential. Without it, nothing works.
• Change management is always underestimated. When a list of predictions arrives at 9 a.m., who acts on it, and how?
• A model can work perfectly and still be useless if the role it assumes doesn't exist in your hospital.
• Measure the effect on real work (time saved, changes to the workflow), not just the model's accuracy.

Being ready

One panelist described an AI-readiness framework for hospitals with five dimensions: strategy (start from a problem, not a technology), literacy, data and infrastructure, policy, and impact. Don't wait to be fully ready on all five: start with low-risk uses while you build capability, then take on higher-risk ones as you mature.

Who owns the result?

When AI makes something measurable for the first time, who is accountable: frontline staff, the unit manager, the quality team? New measurements also raise fears about privacy and jobs, so be clear from the start about how roles will change.

Quiet errors, automation bias and deskilling

• What worried the panel more than a system crash were errors that happen quietly, without anyone noticing, even in "benign" tasks like scheduling a patient's follow-up.
• Automation bias: people defer to the machine and miss obvious signals.
• Keep monitoring after go-live. Deploying a model with no way to track its accuracy over time is a gap.
• Deskilling: if technology replaces the practice that builds expertise, how does anyone become an expert? Training has to adapt.
• More connected hospitals need redundancy and disaster plans, and cybersecurity is patient safety.

Sustainability

Hospitals measure value well (outcomes against cost) but rarely measure AI's environmental footprint. Suggested steps: treat the environment as a value dimension, give project leads simple tools to measure it, and make deliberate trade-offs when goals conflict. Processing data on the device itself (edge computing) can cut how much data and energy a system uses. Deployed selectively, AI could also reduce waste that healthcare already has.