MiniBox · Autonomous Bicycle Counter

MiniBox is my final project for the APSIO professional bachelor’s degree, built as a pair with Noémie Tandol. It’s one of the projects I’m most proud of, mostly for what it was trying to solve.

Cycling keeps growing, with the pandemic, e-bikes and better bike lanes. But to know how many cyclists go by and in which direction, you have to count them, and there wasn’t really any solid study on it.

The idea was a small standalone box, a camera and a Raspberry Pi counting bikes on their own, cheap enough to put up lots of them. It was meant for local authorities who want to see what their new infrastructure changes, and for groups that promote cycling, like 2 pieds 2 roues or the AF3V. It also had to be open source, so anyone could use it for free and improve it. It’s under the GPL-3.0 license, the code is on GitHub and the thesis is a PDF, in French.

On paper, the idea fits in one sentence. A camera films the street, and a program counts the bikes going by. The hard part is counting each bike only once.

Counting each bike once

To spot bikes in the image, we went with YOLOv5, running on PyTorch. In the thesis, we compared it with the other well-known detection models. Faster R-CNN is accurate but slower, and RetinaNet is mostly good with small, packed objects, like in satellite images. YOLO looks at the whole image in a single pass, fast enough for real time, which is what a Raspberry Pi needed.

The catch is that YOLO handles each frame on its own. It sees a bike in one frame, then a bike in the next, without knowing it’s the same one. On its own, it would count a bike as many times as it shows up on screen.

So each bike has to be followed from one frame to the next, with a number that sticks to it. We started with SORT, but the tracking wasn’t reliable, so we moved to Norfair, a lightweight library made to track several objects in real time. Between the two, we needed a small adapter, because YOLOv5 returns its detections as a PyTorch tensor while Norfair expects its own objects, with the box, the confidence score and the class. A bike is counted only once, when the tracker loses it, usually because it has left the frame.

For the direction, each tracked bike keeps its last four positions. The program works out an average movement and, if the bike moves enough, puts it in one of the four directions of the image, up or down, left or right. The threshold is there to ignore things that barely move.

With the display turned on, you can see what the program is tracking. Each bike gets its box, with its number, its direction and the model’s confidence score.

Every minute, the program writes the number of passages per direction and per object type to a CSV file, and once a day it moves that file into a SQLite database. To read those numbers, we built a small Django site. You give it the CSV and it draws the passages with Chart.js, with filters by date and by object type. We picked Django over a plain page so we could add a database or user accounts later on.

Keeping no images

The program keeps no images, and that wasn’t the case at first. The first version took photos at regular intervals and saved them, then another program looked for bikes in them. It was slow, the results file didn’t say much, and we were storing photos of the street.

Filming the street obviously raises legal questions, so we dug into it, and it’s less simple than you’d think. In France, only public authorities such as city halls are allowed to film public roads. A business can’t, and neither can a private person, not even to keep an eye on their car parked outside. The CNIL, the French data protection authority, still sees a camera counting pedestrians, cars and bikes for urban planning as legitimate, but it needs the approval of public services when the people filmed can’t object. And the GDPR doesn’t apply if nobody can be recognized in the video, directly or indirectly.

So in the next version, images go straight from the camera to the model, without ever being saved, and the program only keeps counts. Detection got faster along the way.

How it did

To check the count, we took videos of parks and busy streets and watched them ourselves first, to know what the program should find. On a video just under four minutes long, with quite a lot going on, runners and cars included, we had counted 20 bikes by hand. The program found 21. Directions were less accurate, with a few bikes put on the wrong side. And it processed the video a bit faster than its own length, but with an Nvidia graphics card.

On the Raspberry Pi, installing it turned out to be more painful than expected, between the dependencies and the Python version to stick to. The tracking had its misses too. When it follows cars and bikes at the same time, it sometimes gives a car’s number to a bike, and the other way around. And we never got to what we were aiming for at the start, telling whether a bike is loaded or whether the rider wears protective gear.

My first open source contribution

For the tracking, we had a lead, YOLOv8, which does detection and tracking in one go and would have replaced Norfair. When I tried it, I hit an error as soon as I started tracking. It came from lap, a library it depends on, which didn’t work with our Python version on Windows.

I opened an issue on the Ultralytics repository. The same day, they suggested replacing lap with a SciPy function, and I posted a version in the issue. But we never managed to use YOLOv8 in the project.

It was the first time I reported a bug and suggested a fix on an open source repository. It was a first step into that world, and a way to give something back to a project I was using. For the record, I didn’t do everything in the right order, and my fix didn’t really solve the problem, since their tests didn’t give the same results as with lap. But at least it helped flag it.

The two of us

We did all of this as a pair, through pull requests. I mostly wrote the detection part, Noémie reviewed and approved, and she built the Django site. Unit tests ran on GitHub on every commit and pull request to develop and master, so we wouldn’t break one feature while fixing another. And we regularly showed our progress to our tutor, to adjust as we went.

Where the project stands

The project stopped with the thesis, and the repository is archived. In the thesis, we wrote down what would come next: training our own model to recognize an e-bike or someone carrying a bag, trying Grounding DINO, which finds in an image whatever you describe in a sentence, like a person in a yellow vest, and building a Docker image to install everything on a Raspberry Pi without fighting the dependencies.