Neural networks are powerful stuff, but very voracious. Engineers at the Massachusetts Institute of Technology (MIT) have succeeded in developing a new chip that cuts the power consumption of the neural network by 95%, which could in theory allow them to work even on mobile devices with batteries. Smartphones are getting smarter and smarter these days, offering more AI-fueled services like virtual assistants and real-time translations. But usually neural networks process data for these services in the cloud, and smartphones only transmit data back and forth.
This is not ideal because it requires a thick communication channel and assumes that sensitive data is being transmitted and stored outside the user's reach. But the colossal amounts of energy required to power neural networks powered by GPUs cannot be provided in a device powered by a small battery.
MIT engineers have developed a chip that can reduce this power consumption by 95%. The chip drastically reduces the need to transfer data back and forth between the chip's memory and the processors.
Neural networks are made up of thousands of interconnected artificial neurons arranged in layers. Each neuron receives input from several neurons in the underlying layer, and if the combined input passes a certain threshold, it transmits the result to several neurons above. The strength of the connection between neurons is determined by the weight that is established during the training process.
This means that for each neuron, the chip must extract the input for a specific connection and the weight of the connection from memory, multiply them, store the result, and then repeat the process for each input. A lot of data travels here and there, and a lot of energy is wasted.
The new MIT chip eliminates this by calculating all inputs in parallel in memory using analog circuitry. This significantly reduces the amount of data that needs to be overtaken and results in significant energy savings.
This approach requires the weight of the connections to be binary, not a range, but previous theoretical work has shown that this will not greatly affect the accuracy, and the scientists found that the results of the chip differed by 2-3% from the usual version of the neural network operating on a standard computer.
This is not the first time scientists have created chips that process processes in memory, reducing the power consumption of a neural network, but this is the first time this approach has been used to operate a powerful neural network known for its image processing.
Promotional video:
"The results show impressive specifications for the energy-efficient implementation of rolling operations within the memory array," said Dario Gil, vice president of artificial intelligence at IBM.
"This definitely opens up possibilities for using more sophisticated convolutional neural networks to classify images and videos on the Internet of Things in the future."
And this is interesting not only for R&D groups. The desire to put AI on devices like smartphones, home appliances and all kinds of IoT devices is pushing many in Silicon Valley towards low-power chips.
Apple has already integrated its Neural Engine into the iPhone X to power, for example, facial recognition technology, and Amazon is rumored to be developing its own AI chips for the next generation of Echo digital assistants.
Large companies and chip makers are also increasingly relying on machine learning, which forces them to make their devices even more energy efficient. Earlier this year, ARM unveiled two new chips: the Arm Machine Learning processor, which handles general AI tasks from translation to face recognition, and the Arm Object Detection processor, which detects, for example, faces in images.
Qualcomm's newest mobile chip, the Snapdragon 845, has a GPU and is heavily AI-driven. The company also unveiled the Snapdragon 820E, which should work in drones, robots and industrial devices.
Looking ahead, IBM and Intel are developing neuromorphic chips with architecture inspired by the human brain and incredible energy efficiency. This could theoretically allow TrueNorth (IBM) and Loihi (Intel) to carry out powerful machine learning using only a fraction of the power of conventional chips, but these projects are still highly experimental.
It will be very difficult to force the chips that give life to neural networks to save battery power. But at the current pace of innovation, this “very difficult” looks quite feasible.
Ilya Khel