1. A copilot is not a scribe and not a chatbot
The three are often sold under the same word and they carry completely different risk.
| What it reads | What it produces | Risk if it is wrong | |
|---|---|---|---|
| Chatbot | General knowledge, often not your data | An answer | Low — you can see it is generic |
| Scribe | One visit that already happened | A draft note for review | Moderate — a wrong number in a signed note |
| Copilot | The whole chart, and the clinic’s protocols | A proposed action | High — a plausible wrong plan, confidently placed |
The risk rises because the output moves from description to proposal. A scribe that mishears is wrong about the past, which a provider reading the note can catch. A copilot that misreads a chart is wrong about the future, and it is wrong in the shape of something you were about to do anyway.
2. The rule that makes a copilot safe: it prepares, you press
Every defensible copilot design we have seen reduces to one sentence: the assistant prepares the work and a human performs it. It can assemble a pre-visit brief, find a real appointment slot, pull the right protocol, draft the message. It does not book, send, prescribe or change anything until a person taps.
This sounds like a limitation and is actually the thing that makes the feature usable. The slow part of a provider’s day is rarely the tap; it is the assembling — opening four screens to remember where a patient got to, checking what the protocol said, working out when they are next due. Take that away and leave the decision alone, and you have removed the work without moving the responsibility.
The inverse is the design to refuse. A copilot that books, sends or orders on its own has quietly made a clinical decision under your name, and the audit trail will say you made it.
3. Everything it says should come with a source
A clinical assistant that cannot show where a claim came from is not saving you time, because now you have to verify it yourself.
The test is simple and a demo cannot dodge it: ask the copilot something about a real chart, then ask it where that came from. You should get a specific line — a note, a result, an order, with a date — not a restatement.
| Answer shape | What it tells you |
|---|---|
| “Last A1c was 6.1 on 3 March, from the lab result” | Traceable; you can confirm in one click |
| “The patient has been improving” | A summary of a summary; you cannot check it |
| “Based on their history” | No source. Treat it as unverified |
Insist on the first shape for anything clinical. For scheduling or administrative suggestions the bar is lower, but it should still be able to say why.
4. Where a copilot actually helps
The wins are concentrated in recall and assembly, not in reasoning. Vendors demo the reasoning because it is impressive; clinics get paid back by the recall.
| Task | What the copilot removes | What stays human |
|---|---|---|
| The pre-visit brief | Opening four screens to reconstruct where the patient got to | Deciding what to do about it |
| Booking the follow-up | Working out the right visit type, length and prescriber, then finding a slot | Approving the slot |
| Recalling the protocol | Remembering what your own clinic’s phase three says | Whether this patient should be on it |
| Finding the thread | Searching months of notes for when something started | Interpreting it |
| Drafting the message | Composing the first version | Reading it and sending it |
Notice what is absent: diagnosis, dosing and anything that changes a plan without a person. That is not timidity, it is where the category’s value actually is.
5. Eight refusals to demand before it touches a chart
Ask for these as a written list. A vendor who has thought about this has one already.
| It must never | Because |
|---|---|
| Diagnose | That is a clinician’s act, and a confident wrong one is worse than silence |
| Change a dose | The single highest-consequence number in the chart |
| Book, message or prescribe without a tap | An action taken under your name that you did not take |
| State a clinical fact without a chart source | Unverifiable claims cost more time than they save |
| Compare across clinics | Your BAA covers your patients, not a shared pool |
| Invent a protocol | It should recall yours, not improvise a new one |
| Act on a chart the user cannot open | The copilot must inherit permissions, not bypass them |
| Hide that it is uncertain | A hedge you can see is useful; a hedge you cannot is dangerous |
The seventh is the one most often missed in the build. If a copilot can answer about a patient that the person asking is not allowed to open, it has become a route around your access controls, and the audit trail will not show it as one.
6. How to evaluate one in a demo
Bring a complicated patient, not a simple one. Copilots look excellent on a patient with one condition and one medication, which is not who fills your afternoon.
| Bring | What it exposes |
|---|---|
| A patient with a stopped protocol | Whether it knows the difference between current and historical |
| A patient with an allergy | Whether the suggestion respects it, unprompted |
| A chart with a conflicting note | Whether it picks one silently or surfaces the conflict |
| A question with no answer in the chart | Whether it says so, or fills the gap |
| A staff login with limited access | Whether the copilot respects the same limits |
The fourth is the single most informative test in the list. A copilot that invents an answer to an unanswerable question will do it again on something that matters, and you will not be watching next time.
7. Frequently asked questions
What is an AI copilot for clinicians?
An assistant that reads the patient chart and the clinic’s own protocols to prepare work for a provider — a pre-visit brief, a suggested follow-up, the relevant protocol — without performing the clinical act itself.
How is a copilot different from an AI scribe?
A scribe documents a visit that already happened. A copilot reads the whole chart and proposes what to do next, which is why it carries more risk and needs stricter limits.
Should an AI copilot be able to book appointments?
It should be able to find the right slot and present it. The booking itself should take a human tap, so the action is attributable to the person responsible for it.
What should an AI copilot never do?
Diagnose, change a dose, act without a tap, state a clinical fact without citing the chart, compare across clinics, invent a protocol, see charts the user cannot open, or conceal its uncertainty.
How do I test an AI copilot?
Use a complicated patient: a stopped protocol, an allergy, a conflicting note, and a question the chart cannot answer. The last one shows whether it will invent an answer.