# GPU/TPU vs. evolution

But first, let’s listen to the ode to touching grass:

%[https://www.youtube.com/watch?v=eDr6_cMtfdA] 

Now, to the dull parts.

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">This is not indended to be a direct comparison - rather <strong>estimating the dimensionality and parallel processing demands</strong> for the brain.</div>
</div>

## Paving the landscape

I’ve always thought about the **GPU vs. brain computing** derby - maybe out of sheer enjoyment of those “Top X of…” articles or comparisons, but still.

Also:

* Why there’s so much fuss about wetware computing (mentioned [here](https://posts.teleogenic.com/venture-capital-in-2025))
    
* Where we’ll potentially be headed computing- and storage-wise (e.g. new [Poxiao drive tech](https://www.tomshardware.com/pc-components/storage/worlds-fastest-flash-memory-developed-writes-in-just-400-picoseconds) from China - looks great, but let’s see if it’s adopted in real-life)
    
* <s>Why did the LISP machine fail</s> (they’ve been [outpaced by microchips on their own ground](https://en.wikipedia.org/wiki/Lisp_machine#End_of_the_Lisp_machines))
    
* <s>Why 42</s>
    

And other deep theoretical questions, you know.

First, there are some maxims that I’d like to introduce:

* 🤖 is potentially using a lot less for the ‘infrastructure’ - at the same time, brain seems to only dedicate a small bit (i.e. [10 bps](https://www.technologynetworks.com/neuroscience/news/caltech-scientists-have-quantified-the-speed-of-human-thought-394395)) to high-level thinking
    
* 🧠 is using less energy - [about 0.0125 kWh](https://bond.edu.au/news/how-much-energy-do-we-expend-using-our-brains) compared to [RTX 4090’s 0.45 kWh](https://www.quora.com/How-many-kW-in-total-will-the-Nvidia-RTX-4090-card-consume-if-it-were-working-at-full-power-for-1-month). Otherwise we’d have to get some noice cooling system, boyy! Some are already [researching](https://www.frontiersin.org/journals/neurology/articles/10.3389/fneur.2022.873165/full), as well as [air inlets as T-Rexes did](https://www.science.org/content/article/holes-tyrannosaurus-rex-s-skull-probably-kept-its-brain-cool)
    
    ![Holes in Tyrannosaurus rex's skull probably kept its brain cool | Science |  AAAS](https://www.science.org/do/10.5555/article.2436149/abs/sf-Trex.jpg align="center")
    

Let’s do a comparison on the nature of data transmitted, and the parallelization capabilities.

### Data

![Conputer : r/ChatGPT](https://preview.redd.it/4fok64erllic1.jpeg?auto=webp&s=eae514d011a0b4e4dc5869f6cb96c892f76cb24a align="center")

* 🤖 conputer: **bits** comprising numbers and operations. But in the end, it’s **binary** - 0 or 1. <s>Quantum computing changes that, yeah-yeah.</s>
    
    * in neural networks, we’re definitely operating bigger structures - usually tensors
        
* 🧠 brain: at the first sight it’s binary (neuron either activates or not), yet it’s a lot more complex on the second glance (someone even did [more back-of-the-napkin calculations](https://www.reddit.com/r/MachineLearning/comments/191ol1n/d_human_brain_flops_estimate_is_it_lower_than_we/))
    
    * ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1745411431655/dfa7d383-ff03-4413-adc0-03d940afd6c3.png align="center")
        
        action potential is slightly ‘more’ than binary: it can be [stronger or weaker](https://en.wikipedia.org/wiki/Action_potential#Neurotransmission), at the very least
        
    * ion gradients, i.e. the difference between ion concentrations both sides of the ion pump (they [influence the activity](https://www.sciencedirect.com/topics/biochemistry-genetics-and-molecular-biology/electrochemical-gradient))
        
    * secondary messengers, neurotransmitters, neurosteroids, cytokines, neuron-glia interactions etc. transferring the information inside, between cells and regions. To me, this is the **inflection point** - those transmit and store [a lot of information](https://qbi.uq.edu.au/brain/brain-functions/what-are-neurotransmitters): whether to activate, how strong to activate, how much to spread, which pathways to induce etc. That can be aptly called a [neuron’s metabolome](https://www.reddit.com/r/askscience/comments/izjivy/how_many_bits_of_data_can_a_neuron_or_synapse_hold/).
        
        A simple example - a single messenger, cAMP, can reach ~[40 bits/hour](https://arxiv.org/abs/2408.04988).
        
    * synapses seem to [hold 4.1-4.7 bits](https://direct.mit.edu/neco/article/36/5/781/120323/Synaptic-Information-Storage-Capacity-Measured), a neuron has 7,000 synapses, and we have 86,000,000,000 neurons on average <s>(I know of a SWIM who’s able to write articles with, like, half that!)</s>
        

Let’s highly **theoretically** represent any possible neuron’s data dimension *x* as *d* vector (metabolome and data), information rate as *r* vector, state–synapse weighting as a *W=\[d x s\]* (synapses) matrix, for neuron *i,* synapse *j*:

$$W_j = \left[ d_j \times s_j \right]$$

**Individual information** per synapse:

$$inf_j := W_j \cdot x$$

A-and the **information rate** per synapse:

$$I_j = r_j inf_j(x)$$

Finally we arrive at the possible information throughput per neuron:

$$I_i (x) = \sum_{j=1}^s I_j$$

Despite possible math notation fock-ups, that’s still a **couple of orders more complex than “it fires or it doesn’t”.** For more information, we should cater to [computational neuroscience](https://en.wikipedia.org/wiki/Computational_neuroscience) - I’m no real Pinocchio <s>(at least as of now).</s>

My shabby 5-min drawing skills at it:

![](https://i.imgur.com/vf2dXp6.png align="center")

### Processing

Let’s now consider parallel capabilities.

🤖 The [said RTX 4090](https://www.nvidia.com/en-eu/geforce/graphics-cards/40-series/rtx-4090/) has 16,384 cores, each operating on 2.52 GHz. <s>Okay, I didn’t expect these levels…</s> 41,287,680,000,000 or 4.13×10¹³ operations per second.

🧠 Brain has 86 billion \* 7,000 cores, each operating a lot slower (up to Hz) - but operating **simultaneously**, and always carrying the state/metabolome. Each neuron and, by extension, synapse, is operating **in parallel to all the others**.

### TL:DR

🤖:

* a lot faster per core, yet **synchronous**
    
* homogeneous architecture (mostly; we still have differing core types on CPUs and GPUs, right?)
    
* requires less upkeep e.g. on movement, vision, proprioception, etc.
    
* precise in the sense that everything is low-noise, stored and calculated deterministically
    
* even single-bit errors pose problems and require correction
    
* almost no on-chip re-learning, rather static
    
* state/memory is stored separately, needs transmission
    

🧠:

* a lot more parallelizable and **fully asynchronous**
    
* heterogeneous (neuron/glial cell classes are a thousand and a some - [more than 3000 types at the least](https://directorsblog.nih.gov/2023/10/24/brain-atlas-paves-the-way-for-new-understanding-of-how-the-brain-functions/))
    
* state is higher-dimensional and carries more information, i.e. the [information amounts per ‘cycle’ are incomparable](https://www.beren.io/2023-04-09-GPUs-vs-brains-hardware-and-architecture/). In addition, it’s stored in-cell. Essentially, it’s all in-memory <s>(Redis afficionados here?)</s>
    
* [error-tolerant](https://bmcbiol.biomedcentral.com/articles/10.1186/1741-7007-9-46) via self-repair and redundant transmission. [You can’t lick a badger twice](https://bsky.app/profile/gregjenner.bsky.social/post/3lnhxkdywzc2m), and you can’t fool a neuron
    
* probabilistic in its nature, possibly **quantum** (even more states encoded). UPD: looks like tryptophan complexes [may indeed elicit quantum effects](https://www.thebrighterside.news/post/scientists-discover-quantum-computing-in-the-brain/)
    
* on-line learning and adaptation, ‘weights’ updated dynamically all the time
    

Neuromorphic chips try to mimic the best of brains’ capabilities (namely - parallel processing). And yeah - the reason GPUs and TPUs fare better was precisely the same before optimizations started flowing (more cores working in parallel). More on them a tad later.

One more product I’d love to touch base on is [Green Array Chips](https://www.greenarraychips.com/):

* 144 separate CPUs
    
* each has built-in RAM and can communicate with adjacent cores
    
* energy mostly consumed during computation
    
* ability to continue the computation after power outage
    

Looks a bit more like the brain, eh?

Also, let’s consider the promised <s>land</s> neuromorphic chips, e.g. [Brainchip’s Akida](https://brainchip.com/akida-generations/):

![BrainChip's Akida: Neuromorphic Processor Bringing AI to the Edge | by  NeuroCortex.AI | Apr 2024 | Medium | Medium](https://miro.medium.com/v2/resize:fit:1400/0*BtaLK3VXnbXeo6SD.png align="left")

* Is made of ‘spiking neurons’ (1.2 M of them) connected into ‘synapses’, all organized into 80 nodes with 6 MB on-device memory each
    
* Converts [data to ‘spikes’](https://dl.acm.org/doi/full/10.1145/3571155), i.e. signals that output activation signals able to propagate further via synapses (figure from [here](https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2024.1383844/full)):
    

![Frontiers | Direct training high-performance deep spiking neural networks:  a review of theories and methods](https://images-provider.frontiersin.org/api/ipx/w=1200&f=png/https://www.frontiersin.org/files/Articles/1383844/fnins-18-1383844-HTML/image_m/fnins-18-1383844-g001.jpg align="left")

* Can scale to up to 1,024 chips
    
* They [state](https://brainchip.com/technology/) nodes work independently, but apparently they share the same clock (from [their deck](https://brainchip.com/wp-content/uploads/2019/10/BrainChip-Linley-Akida-Presentation_v5.pdf))
    
* Can perform 🧠 on-chip learning - i.e. changing weights
    

So, these look more or less like the brain <s>(hence the </s> *<s>neuro</s>*<s>morphic, genius)</s>

And that’s my inner mental landscape after trying to understand how virtually anything works:

![Hey, you dropped this- brain v4" Sticker for Sale by zakariadesigner |  Redbubble](https://ih1.redbubble.net/image.3797076752.7339/raf,360x360,075,t,fafafa:ca443f4786.jpg align="center")

### Strategic conclusion

I hope that was okay for a short’eh differences’ snapshot. As always, strategically the thing that counts remains: **which paradigm will be leading in the next 1-2-5-10-30 years?**

IMO there are **a lot of factors** that could influence that:

1. **Research funding** and **breakthroughs** (something becoming real or cheap/reproducible enough)
    
2. **Legal** implications (the legal-illegal/tax-intensive/incentivized spectra)
    
3. Big players **pushing** some paradigm <s>(or a couple to be safe)</s> to the markets, sometimes because of (2)
    
4. Some paradigm becoming or staying a **dead-end** (e.g. because (1) not yet happened)
    

# News and viz’s

### Conputer.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1746090292894/6573eba1-e3a4-4f01-a14d-ec35bf1808a6.png align="center")

### Banana.

Interested in how many people had slipped on banana peels throughout history? Here’s an [investigation](https://www.youtube.com/watch?v=p8W5GCnqT_M).

### Music

![An illustration of a network visualization showing album covers connected by lines.](https://pudding.cool/2025/04/music-dna/assets/tree.jpg align="center")

[Pudding again at it - a pretty awesome insight](https://pudding.cool/2025/04/music-dna/) on how melodies and sounds themselves are <s>plagiarized</s> inherited?

### Best-ial wines

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1746172311124/ecbea12f-98cc-4ca1-85e7-abcfb407d610.png align="center")

Another [awesome Pudding viz](https://pudding.cool/2025/04/wine-animals/) - which animals correlate with higher wine ratings.

**TLDR**: choose fish and bugs, don’t choose lizards.

### Aging like <s>shitty</s> fine wine (movies)

![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ad5cb7e-ecb8-4976-9c65-ff50a014a408_1680x1242.png align="left")

An [exploration of how different movies age](https://www.statsignificant.com/p/which-movies-have-aged-poorly-a-statistical), judged by ratings in two time periods: 1995-2003 and 2018-2023.

TLDR: movies age bad on average; some genres, like horror, age better (below).

![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb132c19e-a50b-4488-8a85-e44ae612c3ae_1816x1234.png align="left")

### Think twice about animals at all…

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1746184217131/5d520ed2-5971-412c-8144-3c54563c3762.png align="center")

They’ve got a [weird (!) effect on life satisfaction](https://link.springer.com/article/10.1007/s11205-025-03574-1) (1) - slightly to the south.

However, as authors mention, that may root in reverse causation: people get pets to deal with loneliness and dissatisfaction.

---

Welcome to **Teleogenic**❣️

Other places I cross-post (not always) to:

* [**Hashnode**](https://posts.teleogenic.com)
    
* [**Medium**](https://baldr.medium.com/)
    
* [**Telegram**](https://t.me/ohmyboi)
    
* [**Twitter**](https://twitter.com/ZakharKogan)
    
* [**LinkedIn**](https://www.linkedin.com/in/zakhar-kogan/)