Real-time transcription with /v1/realtime

The ClovisSTT/realtime model turns streaming audio into text. A WebSocket connection carries audio chunks to the server and transcript fragments to your application.

Endpoint

wss://llm-gateway.clovis-ai.fr/v1/realtime?model=ClovisSTT%2Frealtime

Use an API key authorized for this gateway. The test results described in this guide were obtained on staging.

Authentication uses the Authorization: Bearer <CLOVIS_API_KEY> header when opening the connection. Keep this key on the server: a browser’s native WebSocket API cannot add this header. For a web client, route the connection through your backend.

Two distinct names on the current staging deployment

The URL parameter model=ClovisSTT%2Frealtime selects the public model ClovisSTT/realtime on the gateway. Without this parameter, the test connection was rejected with HTTP 403.

In the session.update event, the backend currently expects voxtral-mini-4b-realtime. Sending the public name returns model_not_found. This distinction was verified on staging; it may change if the gateway adds model-name translation within events.

Prepare the audio

The Realtime protocol expects mono PCM16 chunks at 16,000 Hz, Base64-encoded. Send raw audio samples without a WAV header. An MP3 file, WAV container or WebM recording must therefore be decoded and converted before sending.

Example conversion with FFmpeg:

ffmpeg -i recording.mp3 -ac 1 -ar 16000 -c:a pcm_s16le audio.wav

Session lifecycle

This guide follows the official vLLM Realtime client. It provides an integration starting point for the version deployed on your Clovis gateway.

1. Open the connection and wait for session.created.
2. Select the model with {"type": "session.update", "model": "voxtral-mini-4b-realtime"}.
3. Start processing with {"type": "input_audio_buffer.commit"}.
4. Send chunks with {"type": "input_audio_buffer.append", "audio": "<Base64 PCM16>"} and listen for responses concurrently.
5. Signal the end of the audio with {"type": "input_audio_buffer.commit", "final": true}.
6. Wait for transcription.done before closing the connection.

Received events

Type	Relevant fields	Usage
session.created	id	The connection is established.
transcription.delta	delta	Append this fragment to the displayed text.
transcription.done	text, optionally usage	Use text as the final transcript.
error	error, optionally code	Stop processing and examine the error.

Python example

This example reads a mono PCM16 WAV at 16 kHz, sends 100 ms chunks and receives text simultaneously. It requires Python 3.11 or later.

python -m pip install 'websockets>=14,<16'
export CLOVIS_API_KEY='your-staging-api-key'
python transcribe.py audio.wav

Save the following code as transcribe.py:

import asyncio
import base64
import json
import os
import sys
import wave

from websockets.asyncio.client import connect

ENDPOINT = "wss://llm-gateway.clovis-ai.fr/v1/realtime?model=ClovisSTT%2Frealtime"


async def transcribe(path):
    with wave.open(path, "rb") as audio:
        if (
            audio.getnchannels() != 1
            or audio.getframerate() != 16000
            or audio.getsampwidth() != 2
            or audio.getcomptype() != "NONE"
        ):
            raise ValueError("The WAV must be mono PCM16 at 16 kHz.")
        if audio.getnframes() == 0:
            raise ValueError("The audio file is empty.")

        async with connect(
            ENDPOINT,
            additional_headers={
                "Authorization": f"Bearer {os.environ['CLOVIS_API_KEY']}"
            },
            open_timeout=15,
        ) as ws:
            async def send_event(event):
                await ws.send(json.dumps(event))

            initial = json.loads(await asyncio.wait_for(ws.recv(), timeout=15))
            if initial.get("type") != "session.created":
                raise RuntimeError(f"Session not created: {initial}")

            await send_event({"type": "session.update", "model": "voxtral-mini-4b-realtime"})
            await send_event({"type": "input_audio_buffer.commit"})

            async def send_audio():
                while chunk := audio.readframes(1600):
                    await send_event({
                        "type": "input_audio_buffer.append",
                        "audio": base64.b64encode(chunk).decode("ascii"),
                    })
                    await asyncio.sleep(len(chunk) / (16000 * 2))
                await send_event({"type": "input_audio_buffer.commit", "final": True})

            async def receive_text():
                while True:
                    event = json.loads(await asyncio.wait_for(ws.recv(), timeout=60))
                    if event.get("type") == "transcription.delta":
                        print(event["delta"], end="", flush=True)
                    elif event.get("type") == "transcription.done":
                        print("\n\nFinal transcript:", event["text"])
                        return
                    elif event.get("type") == "error":
                        raise RuntimeError(f"Transcription error: {event}")

            async with asyncio.TaskGroup() as tasks:
                tasks.create_task(send_audio())
                tasks.create_task(receive_text())


if __name__ == "__main__":
    asyncio.run(transcribe(sys.argv[1]))

The 60-second timeout between events is a choice made by this example client; adapt it to your application. For a microphone, replace WAV reading with capture that produces the same PCM16 format and keep receiving events concurrently.

Troubleshooting

Symptom	Check
Connection refused or authentication error	Check the staging address, API key and access permissions.
Model not recognized	Check ClovisSTT/realtime in the URL parameter and voxtral-mini-4b-realtime in session.update.
Invalid audio or incoherent text	Check mono PCM16 at 16 kHz, Base64 encoding and the absence of WAV headers in chunks.
No final transcript	Send the initial commit, then the commit with final: true after the last chunk.
Incomplete text in the interface	Concatenate the delta fragments, then retrieve text from transcription.done.