Excited to share LiveBand, the first tool that enables robust, real-time jamming with an AI audio generator! 💫 🎸 LiveBand runs locally on a Mac, responds instantly to a musician’s input, and can be guided using either audio references or text prompts. The result is a highly versatile musical partner that you can actually jam with. 🚀 It opens up new possibilities for live performance, creative experimentation, and practicing your improvisation skills. We believe that this is a real game changer for human–AI musical interaction! 📜 Paper: arxiv.org/abs/2606.03803v2 🎶 Audio Samples: sonycslparis.github.io/liveb… 📽️ Video: 👇 Made possible by the great Sony CSL Music Team: 👉 @marco_ppasini 👈 (!), @latentspaces , Ben Hayes, @cyranaouameur & @BjareMathias

🎶 New ISMIR 2025 paper! "Estimating Musical Surprisal from Audio in Autoregressive Diffusion Model Noise Spaces" by Mathias Rose Bjare (JKU), Stefan Lattner (Sony CSL), and Gerhard Widmer (JKU/LIT AI Lab). We explore how surprisal — the unexpectedness of musical events — can be modeled directly from audio using autoregressive diffusion models (ADMs). 💡 What we did: - Compared surprisal from diffusion models vs. Generative Infinite-Vocabulary Transformers (GIVT). - Evaluated across tasks: monophonic pitch surprisal (expectation) & segment boundary detection (structural surprise). - Tested surprisal at different noise levels in diffusion processes to see how musical features emerge at multiple granularities. 🔥 Key take-aways: - Diffusion surprisal beats GIVT in modeling pitch expectation & boundary detection. - Mid-level noise surprisal captures pitch-level expectations while suppressing timbre-related variation. - Surprisal curves align with human-like musical segmentation, showing potential as proxies for perceptual surprise. Why it matters: Understanding musical surprisal links computational modeling with human perception and cognition — with applications in AI composition, real-time music interaction, and brain-music studies. 📜 Paper: arxiv.org/abs/2508.05306 💻 Code: github.com/SonyCSLParis/audi… #DiffusionModels #Surprisal #MIR @ISMIRConf @SonyCSLParis @SonyCSLMusic

🌟 New @ieeeICASSP Paper Announcement 🌟 We introduce "Music2Latent2: Audio Compression with Summary Embeddings and Autoregressive Decoding", a novel autoencoder achieving higher fidelity at extreme compression rates using consistency models & summary embeddings. 🧵

😃Accepted @ieeeICASSP papers of @SonyCSLMusic: Accompaniment Prompt Adherence: A Measure for Evaluating Music Accompaniment Systems M. Grachten, J. Nistal Estimating Musical Surprisal in Audio M. Bjare, G. Cantisani, S. Lattner and G. Widmer Hybrid Losses for Hierarchical Embedding Learning H. Tian, S. Lattner, B. McFee, C. Saitis Music2Latent2: Audio Compression with Summary Embeddings and Autoregressive Decoding M. Pasini, S. Lattner, G. Fazekas Zero-shot Musical Stem Retrieval with Joint-Embedding Predictive Architectures A. Riou, S. Lattner, A. Gagneré, G. Hadjeres, S. Lattner, G. Peeters Congrats to the authors! @latentspaces @howariou @GiorgiaCanti @tiianhk @marco_ppasini @GeoffroyPeeters @gaetan_hadjeres @SonyCSLParis

Happy to announce that our legendary DrumGAN, developed 2020 by Javier Nistal @latentspaces is now available as a web application. Free to use and without registration! 🥁 Give it a try at drumgan.csl.sony.fr/ #DrumGAN @SonyCSLMusic

🥳 New publication announcement! Marco Pasini solved the problem of error accumulation in continuous autoregressive models (CAMs), making it possible to generate sequences without the need for prior tokenization. Say goodbye to RVQ codecs (use music2latent 😉). @SonyCSLMusic

🤩 It's exciting to see how Diff-A-Riff makes it easy to build up a song based on some input audio! 👇

😯Surprisal in music is an often underestimated concept. It's not only valuable for music analysis but can also be leveraged to control complexity and structure in music generation. Check out our @ISMIRConf paper, where we use Information Content curves as a control signal for polyphonic music generation: 📜arxiv.org/pdf/2408.06022 Great work by @BjareMathias !🤓 @SonyCSLMusic @cpjku @SonyCSLParis

👩‍🎓@SonyCSLMusic has an open 3-year PhD position starting in spring 2025! We are excited to invite applicants who meet most of the following requirements: - Master's degree - Strong background in (generative) deep learning - Experience with deep learning projects - Publications in peer-reviewed conferences/journals - Musical background - Knowledge in (big) data analysis and management - Excellent programming skills (Python) - Strong problem-solving abilities and creativity - Clear vision or ideas about research topics they would like to pursue - Fluent in English, both written and spoken - Ability to work collaboratively in a team environment Please apply by September 20th with your CV, a cover letter outlining your motivation, research ideas compatible with our research directions, and samples of your work. We strongly encourage applications from members of underrepresented groups. Applications to 📨[email protected]