Computational Creativity in Autonomous Audio Agents: Latent Sound Morphing and Real-Time Procedural Scores
Autonomous generative composition often struggles between two extremes: predictable algorithmic loops or chaotic, unmusical randomness. This paper reports on an autonomous audio agent framework that balances structural coherence with continuous evolutionary surprise. By training deep autoencoder models on diverse microtonal acoustic recordings, our agents navigate continuous latent sound spaces in real time, improvising complex multi-instrumental scores in response to ambient environmental inputs.
1. Autonomy and Agency in Creative Systems
Can software possess creative agency? Rather than automating human composition, we treat computational creativity as the cultivation of machine idiosyncrasies. Our audio agents are given aesthetic constraints, harmonic vocabularies, and listening faculties, then left to improvise open-ended musical works over extended durations.
2. Real-Time Latent Space Resynthesis
Using lightweight convolutional autoencoders optimized for real-time DSP, our system maps raw audio frames into an 8-dimensional latent vector space. The agent navigates this space along smooth spline curves, simultaneously morphing timbre between bowed strings, metal resonators, and vocal phonemes without audible phase artifacts.
3. Self-Listening and Environmental Interaction
Each agent includes an integrated machine listening module that analyzes real-time audio from studio microphones. If ambient room noise increases, the agent modulates its dynamic range and shifts pitch registers to avoid masking. If the room is silent, the agent initiates contemplative, sparse microtonal gestures.
4. Results: 72-Hour Continuous Performance
We analyze a 72-hour continuous public performance of our autonomous agents at the ZIAA Gallery. Spectral analysis revealed sustained formal variety, with emergent musical themes repeating and transforming across multiple hours in a manner reminiscent of complex natural ecosystems.
References & Primary Citations
- O’Connor, M. & Al-Haddad, M. (2025). Latent Sound Navigation in Generative Performance. ZIAA Press.
- Boden, M. A. (2004). The Creative Mind: Myths and Mechanisms. Routledge.
- Deisenroth, M. P. (2020). Mathematics for Machine Learning. Cambridge University Press.