UG JS SDK Docs

This document provides an overview of the ug-core JS SDK, a framework-agnostic JavaScript library for managing real-time, interactive avatar conversations. It is designed to be decoupled from any specific UI framework, allowing for integration into various environments, such as React, Vue, or Vanilla JavaScript projects.

Setup

💡

Beta - Quick demo to start with → https://github.com/uglabs/ug-demos-js

To set up your project locally - you can clone https://github.com/uglabs/ug-demos-js and use that as a baseline - it includes a text like interface, and audio interface including VAD (Voice Activity Detection) support and an Avatar

Host: For Host URL, use: https://pug.stg.uglabs.app

API Key: To get a service account API key, go to https://pug-playground.stg.uglabs.app/service-accounts → click “create service account” → enter it’s name → copy the key

Federated ID: To get a player ID, go to https://pug-playground.stg.uglabs.app/players → click “create player” → enter player’s external id → copy federated id of the player you created (appears at the end of the list)

Install locally


npm install github:uglabs/ug-js-sdk

Live Demo is available here

🏛️ Core Architecture

The SDK is built around a central Conversation Manager that orchestrates several specialized managers to handle different aspects of the conversation lifecycle.

🧞 Conversation Manager: The main entry point and central nervous system of the SDK. It initializes and coordinates all other managers, manages the overall conversation state (idle, playing, userSpeaking, etc.), and exposes a simple API for controlling the conversation.

🗣️ Conversation Network: Manages the WebSocket connection to the backend, handling the sending and receiving of all conversation-related messages.

🎩 User Input Manager: Captures user input. It uses a VAD Manager (Voice Activity Detection) to detect when the user is speaking and an AudioRecorder to capture microphone data. It can also handle text input.

🎪 Playback Manager: Manages the playback of the assistant's response. This includes playing audio (Audio Player), displaying subtitles (Subtitle Manager), and triggering avatar animations (Using Avatar Manager).

✨ Key Features

🗣️ Interrupts (Barge-in) [WIP]

The SDK is designed for natural, fluid conversations. Users can interrupt the assistant at any time, just like in a real conversation. This is achieved through robust Acoustic Echo Cancellation (AEC).
We leverage the browser's native AEC capabilities, which are often hardware-accelerated on modern mobile devices. In environments where hardware AEC is unavailable, a software-based solution is used, ensuring a seamless experience across all platforms.

🤫 Smart Turn Detection [WIP]

No awkward silences or waiting for a push-to-talk button. The SDK uses Smart Turn Detection Model to intelligently detect when a user has started and, more importantly, finished speaking. This allows for a natural conversational flow, where the assistant responds precisely when the user has completed their thought.

UG JS SDK 3.0 🚀 UG-Core SDK Documentation Setup 🏛️ Core Architecture ✨ Key Features 🗣️ Interrupts (Barge-in) [WIP]🤫 Smart Turn Detection [WIP]