Generate realistic sound effects from simple text prompts using Stable Audio, Hugging Face Diffusers, and Gradio.
Prompt2Audio is a Generative AI project that converts natural language descriptions into high-quality audio clips. By leveraging diffusion-based audio models, users can create sound effects such as rain, hammer strikes, environmental sounds, and more with just a text input.
Prompt2Audio demonstrates how Generative AI can transform text into audio using state-of-the-art diffusion models.
Users simply enter a description like:
"Sound of rain falling on a metal roof during a storm."
The model generates a realistic audio waveform based on the prompt.
The project also includes an interactive Gradio interface, allowing users to easily experiment with different sound prompts and durations.
- Python
- PyTorch
- Hugging Face Diffusers
- Stable Audio Open 1.0
- Gradio
- Google Colab (GPU)
- SoundFile
- Mount Google Drive in Google Colab
- Authenticate with Hugging Face
- Install required dependencies
- Load the StableAudioPipeline model
- Provide a text prompt describing a sound
- The diffusion model generates an audio waveform
- Save the generated output as a
.wavfile - Play the audio using the Gradio interface
Prompt:
"The sound of a hammer hitting a wooden surface."
Negative Prompt:
"Low quality"
Output → A 10-second realistic hammer sound effect.
Start the Gradio interface:
python app.pyThis will launch a local web interface where you can generate sounds from text prompts.
- Generate realistic audio from text
- Adjustable audio duration (1–20 seconds)
- Negative prompts for better output control
- Interactive web interface using Gradio
- GPU accelerated inference
Users can:
- Enter a sound description
- Add a negative prompt
- Adjust audio duration
- Generate and listen to the sound instantly
- Diffusion models for audio generation
- Using Hugging Face pipelines
- Building AI interfaces with Gradio
- Running generative models on GPU
Contributions are welcome!
If you'd like to improve this project:
- Fork the repository
- Create a new branch
- Submit a pull request
AKSHITHA HIRAKARI
AI / Machine Learning Enthusiast Passionate about building Generative AI applications