Assembly AI

Assembly AI is an artificial intelligence (AI) company specializing in speech recognition and audio data processing. The company provides speech AI models that enable developers and product teams to build software-based solutions using audio data. These models are particularly used in areas such as speech-to-text transcription, sentiment analysis, summarization, and personal data privacy. Assembly AI is offered as a cloud-based service through a Software as a Service (SaaS) model and operates on Amazon Web Services (AWS) infrastructure.

Founding and General Information

Assembly AI’s headquarters are located in the United States. The company’s CEO and founder is Dylan Fox. Assembly AI has received funding from various investors including Accel, Insight Partners, Nat Friedman, and Y Combinator. As of 2023, the company completed a $50 million Series C investment round. Its customer portfolio includes startups such as Zoom, Supernormal, and EdgeTier, as well as thousands of other startups and enterprise clients across multiple industries.

Core Technologies and Models

Assembly AI stands out with its evolutionarily developed speech recognition models. Key models include:

Universal-1 and Universal-2

These models have been trained on over 12.5 million hours of multilingual audio data. They deliver high accuracy in languages such as English, German, French, and Spanish. The Universal-2 model offers improved performance over its predecessor in areas such as named entity recognition, formatting (e.g., dates, email addresses), numerical data processing, and distinguishing code-switched speech.

Conformer Series

The Conformer-1 and Conformer-2 models achieve high accuracy specifically in English speech recognition. These models combine audio processing with deep learning techniques to better understand complex speech patterns.

Assembly AI offers both asynchronous speech-to-text services for pre-recorded audio files and streaming speech-to-text services that process live audio streams. In the streaming service, latency is maintained below 500 milliseconds while accuracy exceeds industry standards. This service is used in call centers, video conferencing systems, and live event broadcasts.

Audio Intelligence and LeMUR

Assembly AI’s audio understanding layer consists of two core components: Audio Intelligence and LeMUR.

Audio Intelligence provides pre-built models capable of performing the following tasks on audio files:

• Automatic summarization
• Speaker diarization
• Content moderation (e.g., hate speech and sensitive topics)
• Sentiment analysis
• Entity recognition (persons, organizations, email addresses, dates, locations, etc.)
• Personal information redaction (PII redaction)
• Topic detection (according to IAB classification)
• Automatic title generation and identification of key phrases

LeMUR is Assembly AI’s framework for integrating large language models (LLMs) with speech data. This system performs operations such as question answering, text generation, data extraction, summarization, and insight generation via API using speech transcripts. LeMUR is designed to be scalable, enabling processing of large audio datasets with a single API call.

Performance and Security

Assembly AI’s Universal-2 model has achieved word accuracy rates of up to 93.3% in independent evaluation reports. It demonstrates error rates below industry averages on challenging datasets including noisy environments, technical terminology, and accented speech. The platform complies with security and compliance standards such as SOC 2 Type 2, PCI-DSS, HIPAA BAA, and ISO 27001. Users can choose to process their data in European or U.S. data centers, with on-premises deployment options planned for the future.

Pricing Policy

Assembly AI uses a pay-as-you-go pricing model. A free trial provides API access for 90 days. Base pricing for speech recognition varies by model, ranging from $0.12 per hour (Nano model) to $0.47 per hour (streaming model). Pricing for Audio Intelligence features and the LeMUR module is based on per-request charges.

Use Cases

Assembly AI’s products are used across numerous fields including media and entertainment, customer service, medical documentation, sales call analysis, education, content creation, and video subtitle generation. The company integrates with platforms such as AWS, Twilio, and Cloudflare. Additionally, it provides developers with access through its own infrastructure via REST APIs, SDKs, and comprehensive developer documentation.

Future Vision

Assembly AI adopts a research-driven strategy to understand audio data and make speech AI more accessible. Its medium-term vision is to develop “super-human level” speech recognition models that go beyond transcription to deliver understanding, context, and decision-support capabilities. In pursuit of this vision, the company continues investing in expanding both model performance and scalability.