Interface: UnifiedMessageInput

Defined in: packages/core/src/types/media.ts:59

Unified multimodal message input Can contain text, images, video frames, and/or audio

Properties

audio?

optional audio?: MediaAudio

Defined in: packages/core/src/types/media.ts:67

Audio recording - supports live MediaStream

---

file?

optional file?: MediaFile

Defined in: packages/core/src/types/media.ts:71

Generic file input (PDF, CSV, etc.)

---

frames?

optional frames?: MediaStream | Iterable<MediaFrame, any, any> | AsyncIterable<MediaFrame, any, any> | MediaFrame[]

Defined in: packages/core/src/types/media.ts:65

Video frames with timestamps - supports live MediaStream

---

images?

optional images?: MediaImage[]

Defined in: packages/core/src/types/media.ts:63

One or more images (without timestamps)

---

mode?

optional mode?: "transcribe" | "analyze"

Defined in: packages/core/src/types/media.ts:69

Audio processing mode

---

responseSchema?

optional responseSchema?: unknown

Defined in: packages/core/src/types/media.ts:73

Optional response schema for structured outputs

---

text?

optional text?: string

Defined in: packages/core/src/types/media.ts:61

Text prompt or instruction