WASM/ONNX Setup Guide

The UG Labs SDK uses Silero VAD for Voice Activity Detection (VAD), which runs a neural network model in the browser using ONNX Runtime Web. This requires specific binary files to be served from your application.

Why WASM is Needed

Voice Activity Detection (VAD) detects when the user starts and stops speaking. The SDK uses the Silero VAD model, which:

• Runs entirely in the browser (no server round-trips for speech detection)
• Uses ONNX Runtime Web to execute the neural network
• Requires WASM (WebAssembly) for performance

Note: This setup is only required if you're using voice input (inputCapabilities.audio: true). Text-only applications don't need WASM.

Required Files

These files need to be served from your application's static assets at /static/binaries/:

From @ricky0123/vad-web

File	Size	Description
silero_vad_legacy.onnx	~1.9 MB	VAD model (legacy)
silero_vad_v5.onnx	~2.1 MB	VAD model (v5)
vad.worklet.bundle.min.js	~15 KB	Audio worklet processor

From onnxruntime-web

File	Size	Description
ort-wasm.wasm	~5 MB	Main WASM binary
ort-wasm-simd.wasm	~5 MB	SIMD-optimized WASM
ort-wasm-threaded.wasm	~5 MB	Multi-threaded WASM
ort-wasm-simd-threaded.wasm	~5 MB	SIMD + multi-threaded
*.mjs	Various	Worker files for threading

Expected File Location

The SDK expects these files at /static/binaries/ relative to your app's root:

your-app/
├── public/
│   └── static/
│       └── binaries/
│           ├── silero_vad_legacy.onnx
│           ├── silero_vad_v5.onnx
│           ├── vad.worklet.bundle.min.js
│           ├── ort-wasm.wasm
│           ├── ort-wasm-simd.wasm
│           └── ... (other ONNX runtime files)

---

Vite Configuration

Working Example

See copy-wasm-binaries.mjs for a complete working example with the postinstall script already configured.

Option 1: postinstall Script (Recommended)

Create scripts/copy-wasm-binaries.mjs:

import { cpSync, mkdirSync, existsSync, readdirSync } from 'fs'
import { dirname, join } from 'path'
import { fileURLToPath } from 'url'

const __dirname = dirname(fileURLToPath(import.meta.url))
const rootDir = join(__dirname, '..')
const destDir = join(rootDir, 'public', 'static', 'binaries')

mkdirSync(destDir, { recursive: true })

// Helper to find packages (handles npm hoisting)
function findPackageDir(packageName) {
  const locations = [
    join(rootDir, 'node_modules', packageName),
    join(rootDir, 'node_modules/ug-js-sdk/node_modules', packageName),
  ]
  for (const loc of locations) {
    if (existsSync(loc)) return loc
  }
  return null
}

// Copy VAD files
const vadDir = findPackageDir('@ricky0123/vad-web')
if (vadDir) {
  cpSync(join(vadDir, 'dist/silero_vad_legacy.onnx'), join(destDir, 'silero_vad_legacy.onnx'))
  cpSync(join(vadDir, 'dist/silero_vad_v5.onnx'), join(destDir, 'silero_vad_v5.onnx'))
  cpSync(join(vadDir, 'dist/vad.worklet.bundle.min.js'), join(destDir, 'vad.worklet.bundle.min.js'))
}

// Copy ONNX Runtime files
const onnxDir = findPackageDir('onnxruntime-web')
if (onnxDir) {
  const distDir = join(onnxDir, 'dist')
  for (const file of readdirSync(distDir)) {
    if (file.endsWith('.wasm') || file.endsWith('.mjs')) {
      cpSync(join(distDir, file), join(destDir, file))
    }
  }
}

console.log('WASM binaries copied to public/static/binaries/')

Add to package.json:

{
  "scripts": {
    "postinstall": "node scripts/copy-wasm-binaries.mjs"
  }
}

Option 2: vite-plugin-static-copy

npm install -D vite-plugin-static-copy

Configure vite.config.js:

import { defineConfig } from 'vite'
import { viteStaticCopy } from 'vite-plugin-static-copy'
import path from 'path'
import fs from 'fs'

function findPackageDir(packageName) {
  const locations = [
    path.resolve(__dirname, 'node_modules', packageName),
    path.resolve(__dirname, 'node_modules/ug-js-sdk/node_modules', packageName),
  ]
  for (const loc of locations) {
    if (fs.existsSync(loc)) return loc
  }
  return null
}

const vadDir = findPackageDir('@ricky0123/vad-web')
const onnxDir = findPackageDir('onnxruntime-web')

export default defineConfig({
  plugins: [
    viteStaticCopy({
      targets: [
        ...(vadDir ? [
          { src: path.join(vadDir, 'dist/vad.worklet.bundle.min.js'), dest: 'static/binaries' },
          { src: path.join(vadDir, 'dist/*.onnx'), dest: 'static/binaries' },
        ] : []),
        ...(onnxDir ? [
          { src: path.join(onnxDir, 'dist/*.wasm'), dest: 'static/binaries' },
          { src: path.join(onnxDir, 'dist/*.mjs'), dest: 'static/binaries' },
        ] : []),
      ],
    }),
  ],
})

Vite Dev Server Fix

Vite 5+ may block .mjs imports from /public. Add this plugin:

function onnxRuntimeDevPlugin() {
  return {
    name: 'onnxruntime-dev-serve',
    enforce: 'pre',
    configureServer(server) {
      server.middlewares.use((req, res, next) => {
        const urlPath = req.url?.split('?')[0] ?? ''
        if (urlPath.startsWith('/static/binaries/') && urlPath.endsWith('.mjs')) {
          const filename = urlPath.replace('/static/binaries/', '')
          const onnxDir = findPackageDir('onnxruntime-web')
          if (onnxDir) {
            const filePath = path.resolve(onnxDir, 'dist', filename)
            if (fs.existsSync(filePath)) {
              res.setHeader('Content-Type', 'application/javascript')
              res.end(fs.readFileSync(filePath))
              return
            }
          }
        }
        next()
      })
    },
  }
}

---

Webpack Configuration

const CopyPlugin = require('copy-webpack-plugin')

module.exports = {
  plugins: [
    new CopyPlugin({
      patterns: [
        {
          from: 'node_modules/@ricky0123/vad-web/dist/*.onnx',
          to: 'static/binaries/[name][ext]',
        },
        {
          from: 'node_modules/@ricky0123/vad-web/dist/vad.worklet.bundle.min.js',
          to: 'static/binaries/[name][ext]',
        },
        {
          from: 'node_modules/onnxruntime-web/dist/*.wasm',
          to: 'static/binaries/[name][ext]',
        },
        {
          from: 'node_modules/onnxruntime-web/dist/*.mjs',
          to: 'static/binaries/[name][ext]',
        },
      ],
    }),
  ],
}

---

Create React App

CRA doesn't support custom Webpack config without ejecting. Use the postinstall approach:

1. Create scripts/copy-wasm-binaries.mjs (see Vite section)
2. Add "postinstall": "node scripts/copy-wasm-binaries.mjs" to package.json
3. Run npm install

---

Next.js Configuration

// next.config.js
const CopyPlugin = require('copy-webpack-plugin')

module.exports = {
  webpack: (config, { isServer }) => {
    if (!isServer) {
      config.plugins.push(
        new CopyPlugin({
          patterns: [
            {
              from: 'node_modules/@ricky0123/vad-web/dist/*.onnx',
              to: '../public/static/binaries/[name][ext]',
            },
            {
              from: 'node_modules/@ricky0123/vad-web/dist/vad.worklet.bundle.min.js',
              to: '../public/static/binaries/[name][ext]',
            },
            {
              from: 'node_modules/onnxruntime-web/dist/*.wasm',
              to: '../public/static/binaries/[name][ext]',
            },
            {
              from: 'node_modules/onnxruntime-web/dist/*.mjs',
              to: '../public/static/binaries/[name][ext]',
            },
          ],
        })
      )
    }
    return config
  },
}

Install the plugin:

npm install -D copy-webpack-plugin

---

Troubleshooting

"Failed to load .onnx model"

Cause: ONNX model file not found.

Fix:

1. Check files exist in public/static/binaries/
2. Verify files are served: open http://localhost:3000/static/binaries/silero_vad_v5.onnx
3. Check network tab for 404s

"TypeError: Cannot read properties of undefined (reading 'wasm')"

Cause: ONNX Runtime WASM files not found.

Fix:

1. Ensure ort-wasm*.wasm files are in public/static/binaries/
2. Check server returns Content-Type: application/wasm

"Worklet failed to load"

Cause: vad.worklet.bundle.min.js not accessible.

Fix:

1. Verify file exists in public/static/binaries/
2. Check browser console for specific error

"MIME type not supported" (Vite)

Cause: Vite 5+ blocks .mjs imports from /public.

Fix: Add the onnxRuntimeDevPlugin() to your Vite config.

Packages Not Found (npm hoisting)

Cause: npm may hoist packages differently.

Fix: Use the findPackageDir() helper that checks multiple locations.

---

Verification

After setup, verify everything works:

1. Start your dev server
2. Open browser DevTools → Network tab
3. Initialize a conversation with audio enabled
4. Check these requests return 200:
   • /static/binaries/silero_vad_v5.onnx
   • /static/binaries/vad.worklet.bundle.min.js
   • /static/binaries/ort-wasm-simd.wasm

If all succeed, WASM setup is complete.

---

Performance Notes

• WASM files are ~5MB each but loaded on-demand
• Browser caches files after first load
• Files only load when voice input is enabled
• Consider lazy-loading the conversation module for faster initial page load