IBM Research’s lab in Almaden, California announced its prototype NorthPole chip that could make AI on the edge a reality sooner than expected.
Running AI models requires a lot of processing and a lot of energy. The almost weekly announcements of faster, more powerful devices seem unrelenting, but there’s a hard limit approaching. IBM’s NorthPole may help engineers push the limit beyond current technologies.
Moore’s Law states that the number of transistors in integrated circuits doubles every year, with associated improvements in performance. For a long time that law held, but the increase in density has slowed down of late.
The laws of Thermodynamics set a hard limit to how many transistors you could fit into a square nanometer, but it’s the typical layout of chips that hampers performance for now.
The “von Neumann bottleneck” is the biggest culprit when it comes to slowing chips down and making them more inefficient. Von Neumann architecture refers to the typical layout of computer chips with memory on one side, processing on the other, and a data bus connecting the two.
The NorthPole chip is made up of 256 computing units, or cores, each containing its own memory. This hybrid of computing and memory in the same piece of chip real estate eliminates the need for off-chip memory.
Because data doesn’t have to be shifted from memory, to processor, and back to memory again, the chip is a lot faster and more energy efficient.
Brain-inspired chip design
Dharmendra Modha, IBM’s chief scientist for brain-inspired computing, says the cores are connected to each other in a way inspired by the white-matter connections in the cerebral cortex of the human brain.
IBM shared the test results on Science.org which said that “NorthPole achieves a 25 times higher energy metric of frames per second (FPS) per watt, a 5 times higher space metric of FPS per transistor, and a 22 times lower time metric of latency.”
Basically saying that it uses a lot less power, is a lot smaller, and works a lot faster than other chips.
What makes IBM’s initial results especially exciting is that NorthPole was made using 12 nm technology. This means the transistors in the chip are 12 nm big. Current state-of-the-art chip manufacturing technology is already down to 3 nm.
When IBM reproduces NorthPole’s chip architecture at that level, the processing density and efficiencies will be way out in front of anything else the industry currently has available.
Leaps in processing from chips that are smaller and use a lot less power make AI on the edge applications a tantalizing prospect. Autonomous vehicles and even hand-held devices or wearables could eventually run full AI models without the need for cloud computing.
