AI requires a significant amount of electricity to run. In Texas alone, energy demand could increase 5x over the next few years as the state shows signs of becoming an AI data center hub.

Continuing to build data center after data center is not a solution for the electricity demand. A new technology that efficiently processes information closer to where it’s used. Texas Engineers partnered with Taiwan Semiconductor Manufacturing Company (TSMC) to test a type of memory called SOT-MRAM.

A Faster Memory Chip

Photo: University of Texas at Austin

This memory uses magnetic properties that keep information safe even when the power is turned off. Writing data takes just 2 nanoseconds and uses only 2 picojoules of energy. Other memory technologies can take up to hundreds of times longer and use hundreds of picojoules to do the exact same tasks.

The team tested the chips on complex tasks like neural network training and probabilistic graph modeling. Sam Liu, a recent UT Austin Ph.D. graduate and first author of the paper in Science Advances, explained their work.

Advertisement

“The unique combination of speed, energy efficiency and endurance makes SOT-MRAM perfectly suited for AI applications, especially in devices where resources like power and memory are limited,” Liu said. “SOT-MRAM hasn’t been considered for AI hardware since it can only hold two states, but we designed it so we can take advantage of the binary state while still being accurate.”

Thinking on the Spot

This tech could let small devices handle AI tasks right where they are, instead of constantly sending data back and forth to a giant server.

“We show that SOT-MRAM AI accelerators can provide the energy efficiency, with enough accuracy, to eventually replace CPU-based AI accelerators in edge devices such as sensors,” Jean Anne Incorvia, an associate professor at the Cockrell School of Engineering and the project’s faculty leader, explained. “For example, take a robotic hand that senses heat. Just like a human, the local AI in the hand can make a quick decision with enough accuracy to move the hand, without even transmitting the neural signal to the brain. When very high accuracy is needed, then the robot can connect to GPU-based data centers in the cloud.”

The researchers plan to keep refining the speed and efficiency of the chips. Right now, small differences between the individual devices can reduce the AI’s accuracy, so they need to fix that variation moving forward.