How to Integrate the Uberduck API to Generate AI Singing and Rapping Vocals

Developers building automated music apps, virtual avatars, and gaming audio pipelines often hit a wall with browser-based generative audio tools: manual point-and-click interfaces cannot satisfy high-concurrency production workflows or programmatic batch generation. Uberduck provides a dedicated REST API that allows engineering teams to synthesize musical vocals directly in code.

Integrating the Uberduck API for singing and rapping requires three main architectural components: HTTP Basic Authentication, choosing between beat-aligned text-to-rap (/tts/freestyle) or parametric/voice-conversion singing (/speak and /tts/convert), and managing an asynchronous polling loop (/speak-status) to retrieve the rendered audio assets.

1. Authentication and Request Architecture

Uberduck uses standard HTTP Basic Authentication across all REST endpoints. You authenticate requests by combining your API Key and API Secret into a Base64-encoded credential string.

Authentication Setup

1. Obtain your API Key (username) and API Secret (password) from your Uberduck dashboard settings.
2. Format the HTTP Authorization header as:

   Authorization: Basic base64(API_KEY:API_SECRET)
   

3. Set the base endpoint to https://api.uberduck.ai.

import base64
import os

API_KEY = os.getenv("UBERDUCK_API_KEY", "your_api_key")
API_SECRET = os.getenv("UBERDUCK_API_SECRET", "your_api_secret")

def get_auth_headers():
    token = base64.b64encode(f"{API_KEY}:{API_SECRET}".encode("utf-8")).decode("utf-8")
    return {
        "Authorization": f"Basic {token}",
        "Content-Type": "application/json"
    }

2. Programmatic Vocal Synthesis Modalities

Depending on whether your application requires rhythmically locked rap vocals or melodic singing, Uberduck offers three distinct technical pathways:

3. Building an Automated Text-to-Rapping Pipeline

The /tts/freestyle endpoint synthesizes vocals that synchronize with musical tempo (BPM) and beat structures.

Step 1: Discover Available Backing Tracks

You can fetch backing tracks provided by Uberduck to sync your lyrics against known beats:

import requests

def list_backing_tracks():
    url = "https://api.uberduck.ai/reference-audio/backing-tracks?detailed=true"
    response = requests.get(url, headers=get_auth_headers())
    response.raise_for_status()
    tracks = response.json()
    return tracks # Returns list of tracks with uuid, bpm, and metadata

Step 2: (Optional) Generate Rhyming Lyrics

If your audio pipeline automatically generates lyrics from user prompts, you can call /tts/lyrics or supply output from an LLM structured as an array of string arrays (verses and lines):

def generate_lyrics(subject, lines=4):
    url = "https://api.uberduck.ai/tts/lyrics"
    payload = {
        "subject": subject,
        "lines": lines
    }
    response = requests.post(url, json=payload, headers=get_auth_headers())
    response.raise_for_status()
    return response.json() # Returns structured lyric lines

Step 3: Dispatch the Freestyle Request

Submit structured lyrics, target BPM, backing track UUID, and the voice model identifier:

def generate_rap_vocal(lyrics_matrix, bpm=90, backing_track_uuid=None, voice="zwf"):
    url = "https://api.uberduck.ai/tts/freestyle"
    payload = {
        "lyrics": lyrics_matrix,  # Format: [["Line one rhyming phrase", "Line two cadence line"]]
        "bpm": bpm,
        "voice": voice,
        "format": "json"
    }
    if backing_track_uuid:
        payload["backing_track"] = backing_track_uuid

    response = requests.post(url, json=payload, headers=get_auth_headers())
    response.raise_for_status()
    data = response.json()
    return data["uuid"]  # Returns job UUID for async processing

4. Configuring Pitch Timelines and Singing Vocals

For melodic singing rather than spoken-word rap, you can integrate fine-grained pitch timelines or guide-track conversion.

Approach A: Pitch Timeline Synthesis via /speak

When using controllable voice models (which can be checked via GET /voices/{voicemodel_uuid}/detail by inspecting the controls: true property), the asynchronous /speak endpoint accepts pitch curves and phoneme durations:

def generate_singing_notes(speech_text, voicemodel_uuid, pitch_curve, duration_curve):
    """
    pitch_curve: Array of numbers representing target pitch semitones/F0 frequencies across phonemes.
    duration_curve: Array of numbers representing timing durations for each phonetic unit.
    """
    url = "https://api.uberduck.ai/speak"
    payload = {
        "speech": speech_text,
        "voicemodel_uuid": voicemodel_uuid,
        "pitch": pitch_curve,
        "duration": duration_curve,
        "pace": 1.0
    }
    response = requests.post(url, json=payload, headers=get_auth_headers())
    response.raise_for_status()
    return response.json()["uuid"]

Approach B: High-Fidelity Singing via Voice-to-Voice (/tts/convert)

Because fine-grained phoneme-level pitch arrays can be challenging to calibrate from raw text, commercial music pipelines frequently use Voice-to-Voice (V2V). A developer feeds a guide vocal (such as a MIDI guide vocal rendered through a basic synthesizer or a human scratch track) to /tts/convert, adjusting pitch_shift to transpose keys seamlessly:

def convert_guide_vocal_to_singer(guide_audio_url, target_voice_uuid, semitone_shift=0):
    url = "https://api.uberduck.ai/tts/convert"
    payload = {
        "input": guide_audio_url,
        "voicemodel_uuid": target_voice_uuid,
        "pitch_shift": semitone_shift
    }
    response = requests.post(url, json=payload, headers=get_auth_headers())
    response.raise_for_status()
    return response.json()["uuid"]

5. Asynchronous Job Lifecycle and Polling Pipeline

Vocal rendering involves heavy neural inferencing and cannot finish inside a standard synchronous HTTP request timeout. Every synthesis call returns an asynchronous job uuid. Your backend must poll the /speak-status endpoint.

import time

def poll_render_completion(job_uuid, max_attempts=30, delay_seconds=2):
    status_url = f"https://api.uberduck.ai/speak-status?uuid={job_uuid}"
    headers = get_auth_headers()
    
    for attempt in range(max_attempts):
        response = requests.get(status_url, headers=headers)
        response.raise_for_status()
        data = response.json()
        
        # Check completion status
        if data.get("failed_at"):
            raise RuntimeError(f"Vocal synthesis job failed: {data}")
            
        if data.get("finished_at"):
            audio_url = data.get("path")
            return audio_url  # S3/CDN URL containing the generated audio file
            
        time.sleep(delay_seconds)
        
    raise TimeoutError("Audio generation timed out before completion.")

6. End-to-End Orchestration Script

Here is an end-to-end Python pipeline script connecting authentication, dispatch, status polling, and audio download:

import requests
import base64
import time
import os

UBERDUCK_API_KEY = os.getenv("UBERDUCK_API_KEY")
UBERDUCK_API_SECRET = os.getenv("UBERDUCK_API_SECRET")

HEADERS = {
    "Authorization": f"Basic {base64.b64encode(f'{UBERDUCK_API_KEY}:{UBERDUCK_API_SECRET}'.encode()).decode()}",
    "Content-Type": "application/json"
}

def create_rap_track(lyrics, bpm=95, voice="zwf", output_file="vocal_output.wav"):
    # 1. Dispatch rap generation
    print("Submitting vocal generation request...")
    freestyle_url = "https://api.uberduck.ai/tts/freestyle"
    payload = {
        "lyrics": lyrics,
        "bpm": bpm,
        "voice": voice,
        "format": "json"
    }
    
    resp = requests.post(freestyle_url, json=payload, headers=HEADERS)
    resp.raise_for_status()
    task_id = resp.json()["uuid"]
    print(f"Task started with UUID: {task_id}")
    
    # 2. Poll for completion
    status_url = f"https://api.uberduck.ai/speak-status?uuid={task_id}"
    audio_url = None
    for _ in range(40):
        time.sleep(2)
        status_resp = requests.get(status_url, headers=HEADERS)
        status_resp.raise_for_status()
        status_data = status_resp.json()
        
        if status_data.get("failed_at"):
            raise RuntimeError("Generation job failed on server.")
        if status_data.get("finished_at"):
            audio_url = status_data.get("path")
            print(f"Synthesis complete! Asset location: {audio_url}")
            break
            
    if not audio_url:
        raise TimeoutError("Render took longer than 80 seconds.")
        
    # 3. Download the synthesized audio
    audio_data = requests.get(audio_url).content
    with open(output_file, "wb") as f:
        f.write(audio_data)
    print(f"Saved generated vocals to {output_file}")
    return output_file

if __name__ == "__main__":
    bars = [
        ["Kick off the sequence running code in the groove", "Synthesize the bars watch the audio move"],
        ["Hitting every tempo with the BPM tight", "Uberduck API pumping beats through the night"]
    ]
    create_rap_track(lyrics=bars, bpm=92)

7. Operational Best Practices and Technical Trade-offs

When deploying Uberduck into automated production systems, consider these operational realities:

• Syllable Density and BPM Matching: Do not overpack words into rapid tempos. At tempos above 110 BPM, overly dense lyric lines can cause neural TTS phoneme blurring or syllable clipping. Maintain consistent meter across each lyric row.
• Latency vs. User Experience: Synthesis typically takes between 4 to 15 seconds depending on audio length and queue load. For interactive user applications, display progress indicators or process audio via background job workers (e.g., Celery, BullMQ) rather than blocking web request handlers.
• Licensing and Commercial Usage: Commercial integration requires an eligible Uberduck paid subscription (such as Pro or Enterprise). Non-commercial free tiers restrict commercial deployment, and voice clones must adhere to legal attribution and usage terms.
• Stem Post-Processing: The generated output is often returned dry or with minimal master processing. For music-grade apps, run the resulting WAV file through downstream DSP pipelines (equalization, dynamic range compression, de-essing, and spatial reverb) to sit naturally inside the final mix.

References

• Uberduck API Documentation
• Generate Freestyle - Uberduck Text To Speech API
• Generate Lyrics - Uberduck Text To Speech API
• Generate Speech - Uberduck Text To Speech API
• Voice To Voice Conversion - Uberduck Text To Speech API
• Get text-to-speech request status - Uberduck Text To Speech API
• List Backing Tracks - Uberduck Text To Speech API
• Uberduck AI Pricing and Plans