Danh Phan.
PERSONALAI AGENT•Internal Operations

Low-Latency Vietnamese Speech Recognition & Synthesis Pipeline

Self-hosted speech inference cluster optimized for real-time Vietnamese transcription and expressive neural voice generation.

Key Outcome: <350ms end-to-end voice latency
Real-time speech inference pipeline audio waveform visualizer

Quick Facts

Timeline / Year
2026
My Role
Lead Engineer
Industry
Internal Operations
Core Stack
Faster-Whisper, VITS, FastAPI

The Business Problem

Commercial speech APIs imposed high recurring token costs and unacceptable round-trip latency (>1,200ms) for voice interaction prototypes, making real-time voice conversations clunky.

Implemented Solution

Built an optimized on-premise inference pipeline utilizing Faster-Whisper with INT8 quantization for ASR and lightweight VITS models for Vietnamese speech synthesis, achieving sub-350ms streaming latency.

Role & Responsibilities

Machine Learning Engineer: Benchmarked model architectures, converted weights to ONNX/TensorRT, and built streaming WebSocket server.

Architecture & Implementation Details

Engineered asynchronous FastAPI WebSocket endpoints managing streaming audio buffers, Voice Activity Detection (VAD), and token-by-token synthesis piping.

Verified Outcomes & Business Impact

Achieved sub-350ms end-to-end voice latency on a single consumer GPU while saving $400/month in cloud transcription costs.

Related Case Studies

Other projects sharing similar service domains or technology stacks.

Enterprise Agentic RAG Platform
PERSONAL
AI AGENT•AI_AGENT

Enterprise Agentic RAG Platform

Hệ thống AI Agent tự động hóa tra cứu dữ liệu quy chế và phân tích báo cáo tài chính cho doanh nghiệp.

LangGraph, FastAPI, pgvector +2