The Most Interesting Subject in the Universe

by Sergey Nikolenko
Sergey’s a researcher in the field of machine learning (deep learning, Bayesian methods, natural language processing and more) and analysis of algorithms (network algorithms, competitive analysis). He has authored more than 120 research papers, several books, courses “Machine learning”, “Deep learning”, and others. Extensive experience with industrial projects (Neuromation, SolidOpinion, Surfingbird, Deloitte Analytics Institute).
This article compares neurons to machines and explores the human brain’s capabilities and limitations.
Despite decades of steady advances, in many fields the human brain is still more capable than computers. For instance, we handle natural language better — we can read, understand, and parse content from a book. We are pretty good at learning in a broader sense too. So, what does the human brain do and how does it manage to achieve such remarkable results? How do neurons in your brain work differently than transistors in a processor? Naturally, this topic is inexhaustible, but let us try to begin with a few examples.
As you may know, every neuron occasionally sends electrical impulses, otherwise known as spikes, along axons. Neurons never stop and keep sending signals as long as they’re still alive; however, when neurons are “turned off” they rarely send signals. When neurons are triggered, or “turned on”, spikes occur much more frequently.
Neurons function stochastically, meaning they produce electric signals at random intervals. The patterns of these signals can be pretty accurately approximated with a Poisson process. Computers contain logic gates that send signals back and forth, but their synchronization frequency is fixed and by no means random. This frequency is called a computer’s “clock rate”, which has been measured in gigahertz for quite a while now. On every tick, gates on a certain layer send signals up to the next layer. Although this is done a few billion times a second, it’s performed simultaneously, as though the gate were following a strict order.
Actually, it is very easy to see that neurons can synchronize well and count tiny time intervals very precisely. Stereo sound is the simplest and most illustrative example. When you move from one end of the room to the other, you can easily tell, based solely on the sound coming from the television, where you’re going (being able to tell where sound was coming from was crucial for surviving in prehistoric times). You can tell where you’re going by noticing that the sound reaches your left and right ears at different times.
Your inner ears aren’t all that far apart (about 20 cm), and if you divide that by the speed of sound (340 m/s) you get a very short interval — hundredth of milliseconds — between when the sound waves reach each ear. Nevertheless, your neurons pick up on this minor difference excellently, which enables you to figure out precisely where you’re headed. In other words, your brain could process frequency — in kilohertz — just like a computer. Considering the extensive parallel processing performed by your brain, its architecture could generate rather intelligent computational capabilities…but, for some reason, your brain doesn’t do that.
Let us go back to parallel processing for a second. We recognize people’s faces within a few hundred milliseconds, and connections between different neurons are activated within tens of milliseconds, which means that only a few neurons — probably fewer than a dozen — form a serial circuit in the full facial recognition cycle.
On the one hand, the human brain contains an incredible number of neurons, while, on the other hand, it doesn’t have as many layers as a regular processor. Processors have very long serial circuits, while the brain has short and highly parallel circuits. And while a processor core basically works on one thing at a time (but can switch between different tasks with lightning speed, so it appears to you that everything is working at once), the brain can work on a lot of tasks simultaneously, since neurons light up in many areas of the brain when they start recognizing someone’s face or doing some other equally exciting thing.

The illustration above shows how the brain processes a visual signal in time. The light reaches the retina, where it transforms into electrical impulses and then the image is transmitted 20–40 milliseconds later. The first stage takes 10–20 milliseconds (the image shows the cumulative time, i.e. a total of 140–190 milliseconds passes by the time a motor command is issued).
During the second stage, 20–30 milliseconds later, the signal reaches the neurons that recognize simple visual forms. Then there’s another stage, and another, and only during the fourth stage do we see intermediate forms — there are neurons that “light up” when seeing squares, color gradients or other similar objects. Then the brain goes through a few more stages, and neurons capable of discerning high level object descriptions light up 100 milliseconds after the process began. For instance, when you meet someone new a neuron responsible for recognizing her face appears (this is a terrible simplification and we can’t verify this claim but it appears that there is some truth to it). Most likely, a neuron or a group of neurons responsible for this person in general and lighting up whenever you come into contact with her, including when you interact with her not face-to-face, appear. If you see her face again (and the neuron didn’t unlearn or forget this earlier information) that same neuron will be activated ~100 milliseconds later.
Why does the brain work like that? Answering that question with a simple “evolution did it” doesn’t really explain anything. The human brain evolved to a certain point, and that was sufficient to solve problems as we evolved. The rationalist community says that living organisms are not fitness-maximizers who optimize some survival objective function, but rather adaptation executors, who execute “relatively solid” decisions that were chosen randomly at some point. Well, rigid synchronization with a built-in chronometer never took place; however, we can’t tell you exactly why it played out that way.
Actually, in this case, it seems as though asking “why” isn’t all that relevant. It’s better, more interesting, and more productive to ask “how”. How exactly does the brain work? We don’t know for sure but now we can describe the processes going on inside our heads quite well, at least in terms of individual neurons or, in certain instances, groups of neurons.
What can we learn from the brain? First, feature extraction. The brain can learn to make excellent generalizations based on a very, very limited sample size. If you show a young child a table and tell her it’s a table then the child starts calling other tables tables, although they seemingly don’t have anything in common — they could be round, square, have one leg or four. It’s evident that a child doesn’t learn to do this by supervised learning; she obviously lack the training set necessary to do so. One can assume the child created a cluster of “objects with legs on which people place things”. Her brain had already extracted the “Plato’s eidos” and then, when she heard the word for it, she simply attached a label to a ready-made idea.
Naturally, this process can go in the opposite direction, too. Although the neurons (and other things) of many linguists start twitching nervously when they hear the names Sapir and Whorf, one must admit that many ideas, especially abstract ones, are mostly socio-cultural constructs. For instance, every culture has a word similar in meaning to the concept of “love”; however, the sentiment may be very different. American “love” has little in common with that of ancient Japan. Since, generally, all people have the same physiological traits, the abstract idea of “being drawn towards another person” is not simply labeled in language but rather its adjusted and constructed by the texts and cultural data that define it for a person. But let us return to the main point of the article…to be continued next week.