The case
A streaming app knows the song it is playing. OpenCanvas finds that song's official music video, checks that it really is the same recording, and works out how the song's timeline maps onto the video's. Then it keeps the picture on the right second for as long as the song plays.
- Locked to the song. It measures one offset per stretch of the song, so edits and intros do not throw it off, and it never loops.
- Seeks and skips just work. Seek in the song and the video jumps with it.
- Light. It learns everything from the headers of two small MP4 files, about 0.3 MB each.
- Not tied to one app. Any player that knows the title, the artist and where its audio can be read can use it.
It is the engine behind the video in Cosmotune.
Measured, not claimed
0.55 sMedian time to a playable first picture, from a cold cache.
0.95 sMedian time until the song-to-video map is ready.
16 of 16Songs aligned across 6 languages in the test matrix.
The records
- Read
What it does
How OpenCanvas finds the official video and keeps it on the right second.
- Read
How it works
Frame-size envelopes, FFT alignment and the song-to-video map.
- Read
Measured, not claimed
Cold start on 16 songs, with the full tables.
- Read
Use it in your app
Installation, configuration and player integration in one practical guide.