A multimodal piano performance dataset, and the models built on it.
From the MAC Lab at KAIST and the Music Informatics Group at Georgia Tech.
PianoVAM captures piano performance across every modality at once. Each of 107 practice recordings on a Yamaha Disklavier aligns top-view video, lossless audio, ground-truth MIDI, and 21-point hand landmarks to the same take. On top of the dataset we build models that read one modality from another, starting with V2N, which transcribes piano to complete MIDI from silent video alone.
A multimodal piano performance dataset (ISMIR 2025). ๐๏ธ Dataset ยท ๐ฌ Live demo ยท ๐ Paper ยท ๐ป Code
Visual piano transcription from silent video (ISMIR 2026). ๐ฌ Live demo ยท ๐ Paper ยท ๐ป Code
Web toolkits for dataset acquisition and fingering annotation (ISMIR 2025 LBD). ๐ Paper