Block's New AI Model Learns Language From Sound Alone

Jack Dorsey's company Block released a new artificial intelligence model designed to process language without relying on written text. The system, named Dessa, was trained exclusively on audio recordings of human conversation. The company describes it as a true speech-to-speech model, meaning it processes sound waves directly rather than transcribing them into words first.

The development team led by Ragavan Thurairatnam states that the model learned from nothing but human voices. Thurairatnam noted that babies learn to talk without ever seeing a written word, and the team applied that same principle to machine learning. They acknowledged the process is very difficult but confirmed the model succeeded in this approach. The company aims to create a more natural interface between humans and AI by preserving the timing and emotion of spoken language.

Block published several audio samples to demonstrate the model's capabilities. In one exchange shared by Thurairatnam, the conversation flows naturally, though the model initially mistakes the speaker for a person named Ronaldo. Another sample on the company's website features a discussion about Christopher Nolan's films that drifts into a tangent about Bryan Singer's Superman movies. A third clip involves a user describing a surprise trip to Paris, prompting a response the company describes as teary gratitude. When paired with OpenAI's GPT-Live-1 voice model, Dessa produced the most natural-sounding results in the published demos.

The developers argue that current voice assistants flatten human speech into text and fail to match human conversational timing. They claim this approach causes AI systems to become nearly deaf to the emotional cues in a user's voice. The team cited a scenario where a person is in serious pain, but a standard voice AI continues its routine because it cannot detect the distress in the tone. Block positions Dessa as a step toward solving this specific problem.

The model requires substantial hardware to run effectively. Block states that Dessa can operate locally on a MacBook Pro M4 Max or any machine with better specifications. This allows the system to function without sending data to a cloud server, addressing potential privacy concerns for users who prefer local processing.

Block plans to grant early access to a small group of AI safety researchers. The announcement did not include a timeline for a wider public release. The company listed three primary motivations for the project: curiosity, improving human-AI understanding, and safety.