Shush

Shush: Full-Stack WhisperV3 Audio Transcription with Modal & Next.js

Shush is a full-stack demonstration application showcasing how to deploy the high-performance WhisperV3 audio transcription model, enhanced with Flash Attention v2, on the Modal serverless platform. It provides a Next.js frontend for making transcription requests, offering developers a practical example of building scalable, on-demand AI-powered services with efficient backend processing and a modern user interface for interaction.

Features:

• WhisperV3 Model Deployment: Deploys OpenAI's WhisperV3 for accurate audio transcription tasks.
• Flash Attention v2 Integration: Utilizes Flash Attention v2 for significantly accelerated model inference speed.
• Modal Backend: Leverages Modal for serverless, auto-scaling deployment and serving of the AI model.
• Next.js Frontend: Includes a Next.js application for user interaction and submitting audio for transcription.
• On-Demand API: Demonstrates hosting a reliable API that scales according to demand, suitable for variable workloads.
• GPU Acceleration: Configured for A10G GPU usage on Modal to ensure optimal model performance.
• Concurrent Request Handling: Supports a high number of concurrent inputs (e.g., 80) for improved throughput.
• Hugging Face Model Integration: Downloads the openai/whisper-large-v3 model directly from the Hugging Face Hub.
• FastAPI Web Endpoint: Employs FastAPI within Modal to create an efficient and asynchronous API endpoint.
• Audio File Upload: Allows users to upload audio files (e.g., .mp3 format) for transcription via a form.
• Timestamped Transcriptions: Provides transcription output that includes timestamps for better alignment and context.
• Environment Configuration: Guides users on setting up necessary environment variables, like the Modal API URL.
• Python & Bun Setup: Provides clear setup instructions for both the Python-based Modal backend and the Bun-managed JavaScript frontend.
• Auto-Scaling Infrastructure: Implies auto-scaling capabilities inherent to the Modal platform for the backend.
• Network File System Usage: Utilizes Modal's Network File System for temporary storage or handling of audio files.

Summary:

Shush offers a comprehensive example for developers aiming to integrate advanced AI models like WhisperV3, optimized with Flash Attention v2, into a full-stack application. It combines a Modal backend for efficient, scalable AI model serving and a Next.js frontend for a seamless user experience, effectively demonstrating how to build and deploy high-performance, on-demand audio transcription services.