Sabine
patreon
New Computer will Mimic Human Brain -- And I'm Kinda Scared
🕑 Added 2024-01-13 16:00:08 +0000 UTC[This is a transcript with links to references.]
A lab in Australia is building a new supercomputer that will for the first time both physically resemble a human brain, and perform as many operations as possible, about 228 trillion per second.The scary bit is how few operations this are. Yes, how few. Let me explain.
The new supercomputer will be built at the at Western Sydney University. It’s part of a new paradigm called “neuromorphic” computing, that’s computers modelled after the so far best processing apparatuses that nature has given us. BRAINS.
Neuromorphic computing isn’t the same as artificial intelligence. These are two different research areas with different goals. What they have in common are neural networks.
You may know neural networks as the most important software behind artificial intelligence. But while these neural networks do in some sense resemble the neural networks in the human brain, they’re different from the real thing in one very important way. It’s that for AIs, the structure of the neural network is represented in the software, but the physical basis is still transistors lined up on microchips. In the human brain, in contrast, the network is a physical thing. It’s one with the algorithm that runs on it.
This difference is what scientists and engineers want to remove with the neuromorphic computers. These are built to resemble a human brain in terms of hardware, not just in terms of the algorithm that runs on it. This doesn’t mean that they’re built from biological tissue, it just means that the connectivity and functionality of the biological tissue is reflected in the structure of the neuromorphic processor. So the thing doesn’t actually look like a brain, it’s still a box with wires in it, but it works differently than your phone.
The term “neuromorphic” is somewhat vague and encompasses a lot of different things. In some research settings, for example, they basically throw a bunch of wires atop each other to create a network and call that neuromorphic. Some researchers have succeeded with teaching these wires to recognize hand-drawn numbers by adjusting how well the junctions conduct electricity and from there it’s just a small step until they run for president.
Somewhat more sophisticated than a handful of wires is IBM’s “TrueNorth” processor that they first put forward in 2014. It had 1 million cores, each designed to resemble the way neurons fire in the brain, and specifically designed to map the “virtual” neurons of a neural net on the physical ones. IBM has since updated and improved its neuromorphic processors, but they’re still far smaller than the human brain.
The new computer to be built in Sydney is named “DeepSouth” not just to acknowledge its location, but it’s also a nod to Google’s Deepmind and IBM’s TrueNorth. The new computer will however work completely different than either.
The researchers in Sydney want to use what’s called Field Programmable Gate Arrays, that are basically small circuits whose function, in contrast to normal microprocessors, can still be electronically changed. These programmable arrays will take on the function of neurons and simulate in particular the “spiking”, that is the rather sudden threshold at which neurons pass on information. In addition, they’ll be building in some randomness into the behaviour of these artificial neurons because that seems to be playing an important, if not well understood, role in the human brain.
The point of this research project is not to build a super powerful computer. That’d be more difficult because then you’d have to figure out how to train this neural network. For the moment, they just want to find out how the human brain manages to run on so little power. It barely needs 20 Watts to operate, whereas training the currently existing artificial intelligences consumes huge amounts of energy, and running them isn’t cheap either. Sooner or later, this is about to become a problem. Mapping the neural networks from the software to the hardware could dramatically decrease the energy requirement.
Another reason that they use these FPGAs is that they are slow. You see, neurons in the human brain need a few milliseconds or so to update their state because it’s all chemical reactions. You might find that fast, but remember that modern computer processors update billions of times a second. So they’re easily a million times faster than the neurons in our brains. If you want to build something that physically resembles a human brain, therefore, you need slow electronics, so this is what they’ll be doing. The researchers plan to complete the Deep South computer by April this year. It will be remotely accessible for research purposes though I suppose the waiting list will be quite long.
Of course this opens the possibility that once they’ve managed to get this done, they might be able to build a device that works like a human brain but a million times faster which, erm, isn’t scary at all.
Comments
Great story, I had never heard of this. I find it interesting that someone thought of seeing a different use of FPGA fabric than the normal use.
Rad, as Sabine pointed out, neuromorphic computing it is not so much about achieving great computing speed, but the organization of the computing units.
Rad Antonov
Along similar lines, IBM reported a new processor architecture in October dubbed NorthPole. It’s energy efficient and good for inference tasks, like image recognition. Basically, they jammed the processor and memory next to each to avoid moving data around. Corporate piece here: https://research.ibm.com/blog/northpole-ibm-ai-chip Science paper here: https://www.science.org/doi/10.1126/science.adh1174
Neuromorphic computing has been around for some time, as an idea and also as some projects to develop neuromorphic hardware, mostly neuromorphic CPU chips. As to Field Programmable Gate Arrays (FPGA), they have also been around for a while (I am working on a project to develop special GPS and similar GNSS equipment, using FPGA programmable receivers for various applications on Earth Observation, e.g. to monitor variables relevant to climate change, find tsunami before these hit threatened populations by sending prompt warnings, etc.) https://en.wikipedia.org/wiki/Field-programmable_gate_array However, is not clear to me, in Sabine's blog, if people at West Sydney U. are actually developing right now one neuron using FPGA hardware, or a lot of them. Be that as it may, while I wish them best of luck with their project, I think there are more advanced, very well-funded projects to study the brain and develop to some extent (nothing to be scared about) brain-like hardware and software in both Europe and the USA. Two serious approaches at determining the structure of the brain ultimately to better understand brain function and mental illness, including dementia, among other things mostly of interest in medicine: European Union: https://www.humanbrainproject.eu/en/follow-hbp/news/2023/09/26/neuroscientific-breakthroughs-hbp-enabled-fenix-supercomputing-and-data-infrastructure/ US National Institutes of Health https://www.humanconnectome.org/about-ccf https://www.humanconnectome.org As to neuromorphic hardware: https://en.wikipedia.org/wiki/Neuromorphic_engineering Excerpt: "The goal of neuromorphic computing is not to perfectly mimic the brain and all of its functions, but instead to extract what is known of its structure and operations to be used in a practical computing system. No neuromorphic system will claim nor attempt to reproduce every element of neurons and synapses, but all adhere to the idea that computation is highly distributed throughout a series of small computing elements analogous to a neuron. While this sentiment is standard, researchers chase this goal with different methods."
They are not going to succeed. They are missing a fundamental understanding of Multi-Level-Selection and Emergence of evolutionary levels of organization.
