Function: frames()

frames(framesList): UnifiedMessageInput

Defined in: packages/core/src/multimodal.ts:204

Create a frames input for video analysis

Supports static frames or live MediaStream for real-time video analysis.

Parameters

framesList

MediaStream | Iterable<MediaFrame, any, any> | AsyncIterable<MediaFrame, any, any> | MediaFrame[]

Returns

UnifiedMessageInput

Examples

const frames = [{ data: img1, mediaType: 'image/jpeg', timestamp: 0 }, ...];
bot.process(frames(frames));

const videoStream = await navigator.mediaDevices.getDisplayMedia({ video: true });
bot.process(frames(videoStream));