The brief from the coffee outlet was simple: count the cups that go over the counter, so the owner can compare that number with the till. We had seven CCTV cameras already on the wall, a desktop with a mid-range NVIDIA card, and a detection model that could find a cup in a frame without much trouble.
The first version drew a zone over the serving counter and counted cups that appeared in it. On the first full day it reported close to twice as many cups as were actually served.
Why it was wrong
A cup does not appear once. It is put down, picked up, pushed across, sometimes put back. Staff move glasses around while cleaning. A zone counter sees all of that as activity, and activity is not a sale. We had built a good cup detector and asked it the wrong question.
What we changed
We stopped counting cups and started counting handovers. The camera view is split by an invisible line between the staff side and the customer side. Each cup is tracked individually. It counts once, and only when it crosses from staff to customer. A cup coming back the other way does not count, and one that sits on the line is not counted twice.
That fixed the doubling. Then the smaller problems showed up, all found by watching the outlet's own footage next to the system's output:
- A hand passing over a cup made the tracker think it was a new cup. We retuned it to hold on to a cup through short blockages.
- Empty glasses took up to a minute to be flagged. We shortened the detection cycle to a few seconds.
- The boundary we drew from a still image did not match the real counter edge on the live view. We redrew it.
- At one point two copies of the program ran on the same machine and both wrote counts. Now only one can run.
What we took from it
The model was never the hard part. The hard part was deciding what event actually answers the owner's question, and then checking the count against reality, on the client's own footage, until it held up. We now plan that checking time into every vision project from the start.
The full delivery, including how the return is calculated, is in the coffee outlet use case.