From sound
to recognition.
A microphone hears everything. Between the recording and what you see here sits a chain of recognising, filtering and choosing. Here is how it works.
The stations listen without pause. Every sound fragment passes through a recognition model that adds which species it most likely is and how certain it is of that. The basis is BirdNET, the open model for bird sound, which I run on the Raspberry Pi’s through the Nachtzuster fork.
On top of that run almost two hundred models I trained myself. They add the call type: song, contact call, alarm or flight call. So you know not only which bird it was, but what it was doing. Every recording on the site shows its confidence. Recognition is not exact science: a high percentage is probably right, a low one is more of a guess.
A microphone in a garden also picks up people: neighbours outside, a conversation on the street, a phone on the balcony. I don’t want that on the site, and neither do you when it is about you. So every fragment first passes a filter that removes human speech, before anything is published.
This happens in two steps. First a voice detector (Silero) that flags a fragment as soon as it seems to contain speech. The doubtful cases then pass a second model (Whisper) that checks whether it is really intelligible speech. If so, the fragment does not reach the site. Distant mumbling and unintelligible scraps may stay: that belongs to the sound of a village garden, and no one can be recognised from it.
Not every detection gets a recording on the site. Thousands of hits come in each day. If I showed all of them it would be unreadable. So I choose.
At Canopy and Undergrowth only the rare or interesting species get sound on the site. For the bats anything goes, because bat sound is special anyway. And when a common species still yields a fine recording, I can publish it by hand.
In the live ticker at the top of a station page there is a cooldown per species, so one loud blackbird doesn’t flood the list. In the day timeline lower down, repeats are grouped instead. Two mechanisms, for two different goals.
I show how certain a recognition is, and I don’t hide that the system makes mistakes. The raw data is open under CC BY-NC 4.0, and both the system and this site are on GitHub. Anyone who wants to check how something came about, can.
More on this: the open data and API, the technology under the hood, and the story behind the project.