s2s

@alexkroman1/aai/s2s subpath barrel — speech-to-speech, where the whole turn runs service-side.

S2S is the OTHER session mode, and it is opt-in: setting s2s replaces the stt/llm/tts pipeline entirely, so transcription, the model loop and synthesis all happen inside one vendor socket. Two vendors, one shape — each factory returns a serializable DESCRIPTOR ({ kind, options }), and nothing here opens a socket or reads a credential.

Example

An OpenAI Realtime agent

import { agent } from "@alexkroman1/aai";
import { openAIS2s } from "@alexkroman1/aai/s2s";

export default agent({
  name: "Concierge",
  systemPrompt: "You are a hotel concierge. Be brief.",
  s2s: openAIS2s({ model: "gpt-realtime", voice: "marin" }),
});

s2s and the pipeline fields refuse each other at COMPILE time, and so does the top-level voice convenience — an S2S voice rides on the descriptor, because it is the service that synthesizes.

assemblyAIS2s is also on the root barrel, which is the one exception to “provider factories live on subpaths”. S2S became opt-in when the pipeline became the default mode, so the descriptor that opts in sits beside agent() where an author meets it. openAIS2s is on this subpath alone, like every other vendor.

Credentials are never passed here. Each factory’s vendor names the env var its key is read from — ASSEMBLYAI_API_KEY, OPENAI_API_KEY — and the host reads it out of the agent’s own environment when the session starts. That is what keeps a descriptor safe to serialize across the CLI → server → guest boundary. The variable NAMES are not published: an author never types one, and the one case for repointing a stage is apiKeyEnv on the AssemblyAI descriptor, which this stage carries too.

The descriptor type is on the ROOT barrel TOO

S2sProvider — what a factory here returns — is also exported from @alexkroman1/aai, beside the other three stage types, so an agent annotating two stages writes one import rather than two. It stays here as well: this is where the factory that produces one lives. ProviderDescriptor, the base all four narrow, is on the root ALONE now — one interface with four reference pages was three too many.

Functions

openAIS2s()

openAIS2s(options?): S2sProvider

Build an OpenAI Realtime S2S descriptor — the explicit opt-in to speech-to-speech mode on OpenAI’s Realtime API. The API key is resolved host-side from the agent’s env (OPENAI_API_KEY).

Parameters

options?

OpenAIS2sOptions

Returns

S2sProvider

Example

import { agent } from "@alexkroman1/aai";
import { openAIS2s } from "@alexkroman1/aai/s2s";

export default agent({
  name: "Support",
  systemPrompt: "You are a support agent. Be brief.",
  s2s: openAIS2s({ model: "gpt-realtime", voice: "marin" }),
});

Setting s2s replaces the whole stt/llm/tts pipeline.

Interfaces

OpenAIS2sOptions

Options for openAIS2s.

Extends

• ProviderCredentialOptions

Properties

apiKeyEnv?

optional apiKeyEnv?: string

Env var holding this stage’s credential, replacing the provider default. Names a VARIABLE, not a key.

Inherited from

ProviderCredentialOptions.apiKeyEnv

model?

optional model?: string & object | "gpt-realtime-2" | "gpt-realtime"

Realtime model identifier. Default applied by the host (currently "gpt-realtime-2"). Open: the literals autocomplete, any other id compiles.

url?

optional url?: string

Override the WebSocket base URL (testing/proxy).

voice?

optional voice?: OpenAIS2sVoice

TTS voice. Default applied by the host (currently "alloy").

Type Aliases

OpenAIS2sVoice

OpenAIS2sVoice = "alloy" | "ash" | "ballad" | "cedar" | "coral" | "echo" | "marin" | "sage" | "shimmer" | "verse" | string & object

A voice id for the OpenAI Realtime API — one it accepted when this release was cut, or any other string.

OPEN, like every vendor vocabulary here: the voice list is OpenAI’s and grows between this package’s releases, so a voice shipped next week must still compile. The literals are autocomplete, not a guard; an id the API does not know is refused by the API.

References

assemblyAIS2s

Re-exports assemblyAIS2s

---

AssemblyAIS2sOptions

Re-exports AssemblyAIS2sOptions

---

ProviderCredentialOptions

Re-exports ProviderCredentialOptions

---

S2sProvider

Re-exports S2sProvider