SageMaker AI Inference Adds Bidirectional Streaming
🎙️ Amazon SageMaker AI Inference now supports bidirectional streaming, enabling real-time speech-to-text transcription that returns partial transcripts while audio is still being captured. Using the new Bidirectional Stream API, clients open an HTTP/2 connection to the SageMaker AI runtime, which automatically creates a WebSocket to your model container so audio frames and interim transcripts flow continuously. Any container that implements a WebSocket handler per the SageMaker AI contract works out of the box, allowing real-time models such as Deepgram to run without modification. The feature eliminates weeks or months of custom streaming infrastructure work so teams can focus on model accuracy, latency tuning, and agent behavior.
