How to Prepare Character Portraits and Audio for Natural Lip-Sync in Hedra

In AI video generation with Hedra, the realism of a talking character hinges entirely on the quality and preparation of two source inputs: the visual portrait and the driving audio. Hedra’s core avatar engines—notably Hedra Character 3 (hedra-character-3) and Hedra Avatar—treat audio as the operational authority. The model deconstructs the acoustic waveform into phonemes, maps them to anatomical mouth shapes (visemes), and diffuses synchronized micro-expressions across the eyes, eyebrows, jaw, and head pose.

When lip-sync looks rubbery, teeth duplicate, or the face twitches unnaturally during silences, the defect almost never originates in the render settings. It stems from conflicting signals in the initial assets: an open-mouthed source portrait, harsh jaw shadows, acoustic background noise, or clipped vocal dynamics.

This guide outlines the technical preparation standards for character portraits and audio tracks, the exact production workflow within Hedra Studio and the Hedra API, and how to assemble believable, expressive avatar performances.

1. How Hedra Synchronizes Face and Voice

Understanding Hedra's synthesis pipeline prevents common asset preparation mistakes:

1. Acoustic Phoneme Extraction: Hedra analyzes speech transients, vowel formants, and consonant stops directly from the audio file.
2. Viseme Deformation: The model reconstructs the mouth, lips, tongue posture, and jaw line corresponding to the detected sounds.
3. Generative Micro-Expressions: Rather than just animating a 2D mouth mesh, models like Character 3 synthesize contextual head movement, natural eye blinks, eyebrow emphasis, and subtle torso shifting tied to vocal cadence and tone.

Because the neural model uses the starting image as its baseline geometry and the audio waveform as its timing trigger, any ambiguity in either asset directly degrades generation fidelity.

2. Character Portrait Preparation Standards

Your portrait is not just a visual canvas; it is the reference anchor for all anatomical motion. To ensure clean lip-sync and avoid visual tearing, adhere to these rules:

Framing, Camera Angle, and Posture

• Optimal Angle: Use a direct front-facing portrait or a slight three-quarter view (within 0° to 15° of center). Extreme side profiles prevent the model from tracking bilateral lip movement.
• Camera Height: Keep the camera at eye level. Steep bird’s-eye or low worm’s-eye angles compress the jawline and neck, leading to unnatural neck stretching during speech.
• Composition: Frame the subject from the mid-chest or collarbone upward (standard bust or head-and-shoulders framing). Leave comfortable headroom above the hair and clearance below the chin. Do not tightly crop the jawline or crown of the head.

The Baseline Expression: Neutral Rest Pose

• Mouth Closed or Slightly Relaxed: The mouth should be in a resting state—lips touching or parted by no more than one or two millimeters, without exposed teeth.
• The "Smiling Portrait" Trap: Never use an image with a broad smile, open laugh, or pursed lips as your starting portrait. Hedra interprets the resting state of the mouth from the source image. If the character is already grinning widely with visible teeth, the model attempts to deform that open shape into closed consonant visemes (such as /m/, /b/, /p/), resulting in grotesque mouth warping and blurred teeth artifacts.
• Gaze Direction: The eyes should look directly into the camera lens or slightly toward the conversational audience to maintain engaging eye contact.

Lighting and Occlusions

• Even, Diffused Illumination: Ensure balanced lighting across both sides of the face. Avoid high-contrast split lighting, deep chin shadows, or harsh highlights that obscure the perimeter of the lips.
• Zero Occlusions: The mouth, jaw, chin, and eyes must remain completely unobstructed. Eliminate:
  • Microphones, coffee mugs, or hands near the face.
  • Heavy bangs falling over the eyes or across the cheeks.
  • Thick-rimmed glasses casting dark shadows over the eye sockets.
  • Complex collars, scarves, or jewelry overlapping the lower jaw.

Asset Specs and Consistency in Hedra

• Resolution: Use high-resolution images between 1024×1024 and 2048×2048 pixels in PNG, JPG, or WebP format (up to 10 MB).
• Using Hedra Elements: If your character will appear in multiple scenes, register the character as an Element in Hedra (Library → Elements → Create element). Upload up to 10 consistent reference images of the subject and assign an @name. You can then mention @character_name across Spaces and prompts to preserve facial identity across shots.

3. Audio Track Engineering for Flawless Phonemes

Hedra treats audio as the clock and intensity driver. If your audio track has competing acoustic information, the avatar will animate incorrectly.

1. Pure Vocal Isolation (Dry Speech)

• No Background Music or Ambience: Hedra cannot distinguish between vocal frequencies and instrumental chords. Background acoustic guitars, synthetic pads, or city traffic will register as speech energy, causing the avatar’s mouth to flutter, tremble, or stay open during pauses.
• Add Music in Post-Production: Always keep your dialogue tracks 100% dry during generation. Add background music, sound beds, and foley later inside Hedra's timeline editor (Edit Video / Composer) or your NLE.
• Clean Noise Floor: Use noise suppression (such as ElevenLabs Audio Isolation, Adobe Podcast Enhance, or a high-pass filter at 80 Hz) to eliminate room hum, HVAC noise, and air-conditioner hiss.

2. Dynamics, Levels, and Enunciation

• Level Staging: Target vocal peaks between -6 dB and -3 dB Full Scale (FS). Avoid digital clipping (0 dBFS), which flattens waveform peaks into harsh square waves, confusing phoneme detection.
• Avoid Over-Compression: Do not apply heavy radio-style limiters that eliminate natural vocal pauses. The model relies on dynamic drops in vocal energy to trigger natural mouth closures and head resets.
• Articulate Enunciation: Mumbled words or excessively rapid delivery blend syllables together, resulting in soft, generic mouth flaps. Crisp diction yields distinct visemes.

3. Micro-Pauses and Breath Cycles

• Speech without pauses creates an unnatural, robotic avatar that speaks continuously without blinking or resetting posture.
• Leave 300 ms to 600 ms of silence between thoughts, clauses, and sentences. These micro-silences allow Hedra Character 3 to generate organic breathing pauses, head nods, and eye blinks.
• If using Hedra's built-in Speech tool (Manual tools → Speech), use punctuation (commas, periods, ellipses) and synthesis control tags (/) to structure pacing and emotional delivery.

4. Step-by-Step Production Workflow in Hedra

Whether working in the interactive Hedra Agent Space or using direct manual controls, follow this structured process:

[Clean Portrait (Neutral Mouth)] + [Dry Vocal Audio Track (-6 to -3 dB)]
                              │
                              ▼
           [Select Model: Hedra Character 3]
                              │
                              ▼
           [Performance Prompt: Emotion, Posture & Tone]
                              │
                              ▼
           [Render 3–5s Test Pilot Clip] ──► (Verify Sync & Visemes)
                              │
                              ▼
           [Render Full Video (Up to 10 min)]
                              │
                              ▼
           [Hedra Composer (Edit Video): Add BGM & B-roll]

Step 1: Open the Avatar Creation Tool

• Manual Route: In the Hedra console, navigate to Manual tools → Video → Avatar.
• Model Selection: Choose Hedra Character 3 (hedra-character-3). Character 3 offers long-form performance support up to 600 seconds (10 minutes), native multilingual viseme synchronization, and responsive facial micro-dynamics.

Step 2: Attach Inputs and Configure Framing

• Upload your prepared neutral portrait under start_image (or select your saved @character Element).
• Upload your dry voice track under audio.
• Select your target Aspect Ratio (16:9 for widescreen presentations, 9:16 for mobile vertical video, or 1:1 for square feeds) and Resolution (720p or 1080p).

Step 3: Write Actionable Performance Prompts

The prompt in Hedra Avatar does not need to re-describe the character's clothing or hair—the image reference already dictates those. Instead, describe the performance style, mood, and demeanor:

• Effective Prompt Example: "A professional product designer explaining technical specifications, confident and engaging demeanor, subtle head gestures, thoughtful cadence, soft studio rim lighting."
• Expressive Tone Example: "An animated storyteller recounting an exciting memory, expressive eyebrows, energetic head movement, warm smile between sentences."

Step 4: Run a 3-to-5 Second Pilot Render

Before submitting a multi-minute generation, trim a 5-second slice of your audio and run a quick test. Check:

1. Are bilabial consonants (/m/, /p/, /b/) fully closing the lips?
2. Does the character blink naturally during silent pauses?
3. Does the facial texture hold sharpness without flickering around the jawline?

Step 5: Final Editing in Hedra Composer (Edit Video)

Once your character clip finishes rendering, transfer it into Hedra’s timeline editor (Manual tools → Edit Video / Composer):

• Align multiple character shots or scene cuts.
• Layer background instrumental music on a secondary audio track, adjusting its volume to -18 dB to -24 dB so it sits comfortably beneath the dialogue.
• Export the final polished composition as a single master MP4 video.

5. Troubleshooting Common Artifacts

References

• Create an avatar or presenter video - Hedra
• Speech - Hedra
• Run Hedra Character 3 (hedra-character-3) - Hedra
• Give the agent visual context - Hedra
• Edit Video - Hedra