Untether AI, a startup developing custom-built chips for AI inferencing workloads, today announced it has raised $125 million from Tracker Capital Management and Intel Capital. The round, which was oversubscribed and included participation from Canada Pension Plan Investment Board and Radical Ventures, will be used to support customer expansion.

Increased use of AI — along with the technology’s hardware requirements — poses a challenge for traditional datacenter compute architectures. Untether is among the companies proposing at-memory or near-memory computation as a solution. Essentially, this type of hardware builds memory and logic into an integrated circuit package. In a “2.5D” near-memory compute architecture, processor dies are stacked atop an interposer that links the components and the board, incorporating high-speed memory to bolster chip bandwidth.

Founded in 2018 by CTO Martin Snelgrove, Darrick Wiebe, and Raymond Chik, Untether says it continues to make progress toward mass-producing its RunA1200 chip, which boasts efficiency with computational robustness. Snelgrove and Wiebe claim that data in their architecture moves up to 1,000 times faster than is typical, which would be a boon for machine learning, where datasets are frequently dozens or hundreds of gigabytes in size.

High-speed architecture

Each RunA1200 chip contains a RISC-V processor and 511 memory banks, with the banks comprising 385KB of SRAM and a 2D array of 512 processing elements (PE). There are 261,632 PEs per chip, with 200MB of memory, and RunA1200 delivers 502 trillion operations per second (TOPS) of processing power.

One of Untether’s first commercial products is the TsunAImi, a PCIe card containing four RunA1200s. App-specific processors spread throughout the memory arrays in the RunA1200s enable the TsunAImi to deliver over 80,000 frames per second on the popular ResNet-50 benchmark, 3 times the throughput of its nearest competitor. According to analyst Linley Gwennap, the TsunAImi outperforms a single Nvidia A100 GPU at about the same power rating, or about 400W of power.

Untether is shipping TsunAImi samples and aims for general availability this summer. The company says the cards can be used in a range of industries and applications, including banking and financial services, natural language processing, autonomous vehicles, smart city and retail, and other scenarios that require high-throughput and low-latency AI acceleration.

“Untether AI has a scalable architecture that provides a revolutionary approach to AI inference acceleration. Its industry-leading power efficiency can deliver the compute density and flexibility required for current and future AI workloads in the cloud, for edge computing, and embedded devices,” Tracker Capital senior advisor Shaygan Kheradpir said in a press release.

There’s no shortage of adjacent startup rivals in a chip segment market anticipated to reach $91.18 billion by 2025. California-based Mythic has raised $85.2 million to develop custom in-memory compute architecture. Graphcore, a Bristol, U.K.-based startup creating chips and systems to accelerate AI workloads, has a war chest in the hundreds of millions of dollars. SambaNova has raised over $1 billion to commercialize its AI acceleration hardware. And Baidu’s growing AI chip unit was recently valued at $2 billion after funding.

Toronto-based Untether’s total raised now stands at $152 million.