AI School · Level 1 · Practical case · First steps, no risk

Before you send a photo to the chat: the box, the prescription and what shows up behind

Léelo en español →

Reading time: about 11 minutes

The situation. A visitor brings you a box of a medicine bought in another country. You do not recognise the name, the print is tiny and the patient is waiting. You have your phone in your hand and the temptation is obvious: a photo, into the chat, "what is this and what is the Spanish equivalent?".

Taking a photo is the most natural thing in the world, and that is exactly why it is where data slips out without anyone noticing. With text you have already learned not to type the patient’s name; with a photo, what identifies someone is not what you ask but what is around it: another person’s pharmacy label, the prescription peeking out underneath, the computer screen, a reflection. This case is about solving the query — which is legitimate and useful — without sending anything you should not.

What you need to hand: The box, your phone and two minutes before you press the shutter. And the lesson on images and voice, because this case is that lesson put into practice. If the box carries a label with the patient’s name, the photo does not get sent — you copy the text from the box by hand.

Step by step

  1. Decide whether you really need a photo

    The first question is not how to take the photo but whether you need one. To identify a foreign medicine you nearly always need just three things you can type in ten seconds: the active ingredient (it is on the box, even if in another alphabet or small print), the strength and the dosage form.

    If you can read those clearly, type them. Text you type contains exactly what you have decided it should contain; a photo contains everything that was in front of the lens, and you decided none of that.

    A photo makes sense when the text cannot be typed: Cyrillic, Greek, Arabic or Chinese script, or a pack where it is unclear which word is the brand and which is the active ingredient. That is where a photo genuinely saves work, and that is where it has to be done properly.

  2. Isolate the box before you shoot

    Put the box on its own on a clean, empty surface: a cleared counter, a blank sheet of paper. Do not hold it up in front of the computer or on top of the day’s pile of prescriptions. Look at the phone screen before you shoot, not after, and check the edges of the frame, because that is where everything creeps in.

    What usually turns up in a photo taken in a hurry at a counter, in order of frequency: the pharmacy system screen showing another patient’s record, a prescription or a receipt underneath, another pharmacy’s dispensing label stuck on the box itself (with a name and sometimes personal dosing instructions), and somebody’s face or hand.

    If the box has a label with a name on it, cover it with your finger or a piece of paper — or better, photograph only the side of the pack that shows the active ingredient. You do not need the label to know what the medicine is.

  3. Look at the hidden data in the image

    A photo carries more information than you can see. Many phones store the location where it was taken, plus the date and time, inside the file itself. If you send it as it is to an outside service, you have also sent where your pharmacy is and what time you served that person.

    There is no need to panic: most chat apps strip that data on upload, and many phones have a setting to stop the camera saving location. But it is worth knowing it exists and checking your own phone once. It is a thirty-second setting that lasts for ever.

    The cleanest alternative, and the one I recommend for work, is to take a screenshot of the cropped photo. A screenshot does not carry the original location, and it forces you to crop, which is exactly what you wanted.

  4. Ask without telling the patient’s story

    With a clean image, the question does not need anything about the person either. There is no need to say it is for a 78-year-old woman on holiday from Romania with high blood pressure: you do not need that to identify a pack, and every detail you add is one more detail leaving the pharmacy.

    I am attaching a photo of a medicine pack bought outside Spain.

1. Tell me which active ingredient or ingredients it contains, the
   strength and the dosage form, copying EXACTLY what the box says.
2. If there is anything you cannot read with certainty, say so and do
   NOT guess it.
3. Tell me what I should search for in the Spanish medicines database
   to find the equivalent available here (same active ingredient and strength).

Do not present the equivalent as certain: only what you can see on the box.

    Notice that you are not asking for the equivalent but for what to search for. You decide the equivalent yourself in the official database, which is where the reliable information on what is marketed in Spain lives.

  5. Check what it read with your own eyes

    A model reading an image can misread it, just as you can with tiny print. The difference is that you notice and it does not tell you. So, with the answer in front of you, go back to the box and confirm the active ingredient and strength it gave you, letter by letter.

    Watch for three classic confusions when images are read: units (mg versus µg, which look very alike in some alphabets and typefaces), decimal commas and points (0.5 versus 5), and combinations (a box with two active ingredients where only one gets read).

    Then go to the official database with the active ingredient and strength. If that exact strength or combination does not exist in Spain, the AI does not solve that: it is solved with pharmaceutical judgement and, where needed, by referring to a doctor.

A full example, and what is wrong with it

This is the real reply to a photo of a box with Cyrillic text, taken in a hurry on the counter, using the prompt from step 4. The pack is a combination blood pressure medicine. It is shortened.

What it replies

"The pack contains amlodipine 5 mg tablets, 30 units. It is a calcium channel blocker used for high blood pressure.

To find the Spanish equivalent, search the database for "amlodipine 5 mg tablets"; many generics are marketed.

The image also shows an adhesive label with the name "M. Popescu" and the instruction "1 tab in the morning", which appears to be the dosing prescribed to the patient."

And now, what is wrong with that reply:

  1. It has missed the second active ingredient. The box was a combination of two drugs and the reply names only one. If you search the database for amlodipine alone, you give the person half of their treatment. This is exactly the "combinations" confusion from step 5, and you do not notice unless you look at the box again.
  2. The photo carried the patient’s name, and the model read it. The dispensing label was in the corner and whoever took the photo did not see it. The model did: it transcribed it in the reply. That name has already left the pharmacy, whether or not it appears in the answer. That is why step 2 comes before the question.
  3. It never said "I cannot read this with certainty", even though the second name was in small print and partly covered by the label. Instead of flagging what it could not see, it answered only what it could see as if that were everything.
  4. It gave the equivalent as settled ("many generics are marketed") when it was only asked what to search for. That is not serious here because amlodipine is marketed, but it is the habit to watch: it answers more than it is asked.

Two privacy failures and two content failures in a reply that, read quickly, looks like perfect help. And the most serious is not about content: a photo taken in ten seconds carried a name and personal dosing instructions that nobody meant to send.

The good version of this query costs one more minute: box on its own, label covered, cropped screenshot. And looking at the box again when the answer arrives, which is what would have found the second active ingredient.

And if your pharmacy is not like that

If what you want to photograph is a prescription. Do not. A prescription is a document with health data about an identified person, and there is no way to crop it clean: the name, the diagnosis implied by the medicine and the doctor all come together. If you are unsure what it says, type the medicine and the dosing, nothing else. If the problem is the doctor’s handwriting, you phone the doctor.
If you work in a pharmacy with lots of tourists. This will happen every week, so it is worth building a routine: one corner of the counter kept clear for photographing packs, the phone camera set not to save location, and a note with the prompt from step 4 saved. In high season, the rush is what lets the labels slip in.
If you are handed a loose blister strip with no box. That is harder and riskier: a blister carries less information, sometimes only the brand. If the active ingredient cannot be read clearly on the blister itself, identification by photo is not reliable and you should tell the patient so. An "I think it is" about a blister with no box is not something you dispense on.

When it does not work first time

The chat tells me it cannot read the image properly.
That is the best answer it can give you, not a failure. Take another photo with more light, closer and without reflections, or type the letters you can make out. If it still cannot be read, do not force it: find another route.
I sent the photo and then saw a label with a name on it.
Delete that conversation, and if the service lets you delete uploaded images or history too, do that. It does not fully undo what has already left, but it reduces what is kept. And next time, look at the screen before you shoot: that is the only moment it can be prevented.
It gives me an active ingredient I have never heard of.
It may be a drug not marketed in Spain, or it may be a misreading. Before concluding anything, compare it with the box letter by letter and look it up in an official source. If it appears nowhere, the most likely explanation is that it misread.
The patient asks me to check it "with the phone thing" in front of them.
Do it with the box on its own and show them what you are doing: you cover the label, you crop. People are reassured to see you look after their data, and it is a good moment to explain that you confirm the equivalent yourself in the official database, not the machine.

Before you use what it gave you

It reads what you did not ask about. A model looking at an image reads everything in it, including what you had not noticed. If there is a name in a corner, it has it, whether or not it transcribes it.
It silently leaves out what it cannot see well. Instead of saying "there is a second name I cannot read", it answers with what it can read as if that were all. Combination products are the usual victims.
It mixes up units and decimals. mg and µg, 0.5 and 5. In an image with an unusual typeface or low resolution this is easier than it seems, and the error is a factor of ten or a thousand.
Now, with your own. Take any box from the back room that carries no personal label and do the whole exercise: isolated photo, cropped screenshot, the prompt from step 4 and a letter-by-letter check. Do it now, without any rush, so that on the day the customer arrives with the foreign box you already have the habit and do not have to think about it.

← All practical cases Back to the school →