cielbird

Designing a 12 bit processor

2026-04-26T00:00:00+00:00

I wanted to post this beauty here.

This is a 12 bit processor, designed from the ground up, with the help of a friend. It's my first time designing an integrated circuit this large and complicated.

What you're seeing is the layout for fabrication on silicon on a 180nm TSMC process. Each most colors here are different metal interconnect layers. Below them in blue are TSMC's logic cells. The inputs and outputs go off the screen. Those are the data and address bus lines.

I'm pretty satisfied with the performance. With my limited setup I was able to design it for a whopping 28.6MHz clock speed ! That takes into account all sorts of delays due to logic cell capacitances and interconnect parasitics.

And yes, it works. Here's a simple program running :

Modeling a conlang's phonology

2026-04-25T00:00:00+00:00

Context</h1>
I've worked on making a couple fictional languanges in the past. They're known as conlangs</a>, and there's a whole online community around the art of constructing fictional languages.
Something people like to do is to "evolve" their languages to create dialects or descendent languages. You apply phonological and grammatical changes to the language to create a new one, much like real languages change over time.

Geþeode: the project</h1>
After tediously applying changes to words in my fictional languages, I wanted to make software to automate the process. I called this project "geþeode", after an Old English word meaning language.
I started with phonological changes, or sound changes, as they are pretty predictable. They can also be pretty simple. The same sound changes often occur across different languages and in different time periods. As an example, here's a change that happened from classical latin (~200 BCE) to vulgar latin (~500 BCE) :

kʷ → ɡʷ / V_V </blockquote>
This is a pretty standard notation. Here it means a "kw" sound became voiced when it occurred in contexts between two vowels (V). Words like "aqua" [ˈaɡ.wa] became pronounced more like "agua" [ˈaɡ.wa]. Modern Spanish "agua" [ˈaɣ.wa] inherited from this change.
If you're unfamiliar with this ([ˈaɣ.wa]) notation, it's the International Phonetic Alphabet</a>. Parsing IPA notation was a large part of this project.
An important part of this was to stay close to real linguistic notation. I didn't want to invent my own notation, which is something most other tools do. For reference, I used this amazing compilation</a> of sound changes.
Development</h2>
Being a performance obsessed low-level software guy, I decided to code everything in Rust.
I used `nom</code> for parsing, which has been such a great experience. I will never write my own parser again in rust !`
`Universal phonological model</h2>`Linguists can't agree on how language sounds exist and work in our brains, so there's no "universal" phonological model. However, there's concensus on enough stuff for me to pick a middle ground. Linguists agree that all phonological sounds (called segments) can be defined as sets of distinctive features</a>. Either the feature is present (+) or not (-), so they're binary. Most other sound change tools like SCA2</a> require you to essentially define the phonological model you use. They don't define a universal set of phonological features, and they're basically fancy text editing engines. For many cases, it's quite enough. A pretty defining trait of my tool is that I assume a certain phonological model, and try to accommodate for all use cases. Trees, not strings</h2> Most features define individual segments. However, some features describe syllables, and some describe groups of syllables (called feet). These are suprasegmental features</a>, and are usually things like stress, or tone. They're actually pretty few in number compared to regular segmental features, so it's easy to put them aside. All the sound change software tools I've found ignore these, or deal with them in hacky ways. But if you account for them, the model of a phonological sequence completely changes. It goes from a simple string to a tree. Here's "bailar" (Spanish, to dance) decomposed for illustration : [ _ ] < word or foot level features / \ [-stress] [+stress] < syllable level features | | | | | | [b] [a] [ɪ] [l] [a] [ɾ] < segment level features</code></pre> The model is a tree, where each node has a set of features relative to it's depth, and where each leaf is at the maximum depth. Some questions are hard to answer : How many levels in the tree ? Can sound changes consider prosodic foot</a> ?</li> </ul> </li> Which features should exist ? What binary features should be used to describe different rising/falling tones ?</li> </ul> </li> </ul> Rules</h2> Now since the phonological sequence is a tree, sound changes must be interpreted as tree operations. Getheode actually compiles sound changes into a set of search and replace operations that not only match segment features, but also surrounding and internal tree structure. This model allows for much easier application of sound changes. For example, the following rule only applies in certain word-boundary contexts : t -> d / V#_V </blockquote> Here, a [t] sound becomes voiced, becoming [d], when it is word-initial (# indicates word boundary), and where the surrounding sounds are vowels. Branching</h2> Getheode also supports optionals or branches in the rule. For example : {n,q,h} -> Ø / _(s)a </blockquote> Here, either a [n], a [q], or a [h] are removed, before a [a] or a [sa]. This rule is imaginary and very unlikely. But it works in Getheode. Tagging</h2> The tool supports manually tagging features. For example : V_0ʔV_0 -> Vː_0 </blockquote> Here, we want two vowels (V) that are the same (both have _0) that surround a glottal stop (ʔ) merge into one long (ː) vowel , removing the glottal stop. (The glottal stop is the sound you make when you go "a, a, a" = [ʔaʔaʔa]) Under this sound change, [aʔaʔila] becomes [aːʔila]. The largest issue</h2> Many sound change rules ignore syllable and word boundaries. However, a rule's output cannot be ambiguous, and it cannot be easily inferred. So, this means that some rules are interpreted more strictly than they should, which I'll talk about in a different post. Conclusion</h2> The project's page is here</a>. Most rules are applied correctly. I will make a WASM web demo soon. Next, I will tackle phonotactics</a>, which are the rules that govern how phonemes can be arranged in words in different languages. I'd like to apply a set of sound changes on not only a set of words, but a generalized phonotactic rule set.

Distributed AI in Rust

2025-09-01T00:00:00+00:00

Introduction</h1>
AI models and their training data are big. Training an AI model on only one GPU can become slow, and many models don't even fit on one single GPU. Many techniques exist to manage the memory and speed limitations of modern models. These techniques have unlocked many of the impressive advances in machine learning we see today. These training and inference techniques all depend on basic building blocks of collective communications called collective operations.
In this blog we will cover collective operations, how they are used to speed up training. We'll also discuss the unique way Burn</a> implements collective operations in the 0.20.0 release, as well as how you can easily train your models on multiple devices and multiple nodes.

Data Parallel Training</h1>
Distributed data parallel training (DDP) is a basic example of distributed training. It consists of splitting the training data between multiple devices and training a copy of the model on each device, all while keeping the parameters in sync.
First, the training data is split between each device. Then, during each training step, each device does a forward and backward pass on its own batch of data. The resulting gradients are aggregated between each device. Each device then optimizes its model with the new gradients. At this point the models have the same optimized parameters.
Figure 1: Data distributed training on three devices
This technique allows training time to be cut down significantly, as long as the gradient syncing is negligible. This technique still requires each device to store the entire model in memory, which is an issue tackled by other distributed training techniques.
It is clear that the key to this technique is the gradient syncing. The gradient syncing must be as efficient as possible as to not be a bottleneck in the pipeline.
^{1
The term 'devices' refers to GPUs, TPUs, and other computational hardware commonly utilized in machine learning applications.
</div>}Collective operations</h1>
The syncing of gradients in a data parallel training is a collecive operation called an all-reduce. An all-reduce is one of many primitive collective operations. Some others are:

broadcast: one device sends a tensor to all others</li>
reduce: a tensor on each device is reduced to one tensor on one device</li>
reduce-scatter: a tensor on each device is reduced, each device ends up with a part of the resulting tensor</li> </ul>
PyTorch and TensorFlow don't implement their own collective operations, instead they make use of communication libraries such as NCCL</a> (for NVIDIA GPUs), MPI, or Gloo.
NCCL is the library used for NVIDIA GPUs. It abstracts collective operations using protocols like NVLink, PCIe, GPUDirect RDMA, and even TCP/IP. For all of NCCL's benefits, it is only useful for NVIDIA devices, which goes against Burn's core principles.
Moreover, Burn already has the tools for tensor communication between devices on the same machine with `Tensor::to_device</code>. We can take advantage of shared memory, or even backend specific protocols like NVLink</code> for an Nvidia backend. Logically, GPU-to-GPU communication on the same machine should be done with to_device</code>.`
`For these reasons, we decided to implement our own collective operations crate called burn-collective</code>. For intra-node communication, we use Tensor::to_device</code>, taking advantage of all the backend specific optimisations. For inter-node communication, we use TCP/IP. This two-step separation will show up later.`
`Figure 2: Pytorch and NCCL compared to Burn`
How Burn implements collective operations</h1> Burn currently only supports all-reduce. Reduce and broadcast are also supported, although only in single-node contexts. How many processes?</h2> We started with all-reduce</code>, because it is the backbone to data distributed training. With PyTorch, you usually assign a different process to each GPU. There are many reasons for this, but a big one is the Python's Global Interpreter Lock (GIL). The GIL only allows one thread to hold the Python interpreter at a time, which essentially prevents anything written in Python to actually be multi-threaded. Thankfully, we're not using Python. As said before, we can use to_device</code> to take care of intra-node communication. We can assume the user will launch a thread for each GPU. So for one machine, we only need one process. Figure 3: Burn collective: an example structure with 4 peers and 2 nodes Local and global</h2> Since intra-node and inter-node communication are fundamentally different, we decided to split collective operations between a local (intra-node) and global (inter-node) level. Internally, the algorithms are implemented differently on the internal level and global levels. This leads to a process-per-node structure. It is worth noting that the local/global separation is an implementation detail, and it is only necessary to know when configuring the collective. From a user's perspective, all the other peers, whether on the same node or not, are just as accessible. Walkthrough of an All-Reduce</h2> Lets walk through the internals of a call to all_reduce</code> Each thread must first register, passing a CollectiveConfig</code> that contains information about the number of peers on the same node, as well as the number of nodes in the collective. The call to register</code> is blocking, so it syncs all the threads. When the node's LocalCollectiveServer</code> has registered each peer on the node, the node will register on the global level if necessary. Then, on the global level, the GlobalOrchestrator</code> acts as a rendez-vous point for each node. After registering, the nodes have the addresses of every other node in the collective, and they can be as independent as possible. In the future, the GlobalOrchestrator</code> could allow for a dynamic topology, keeping nodes updated on any changes. Next, all peers in the collective call an all_reduce</code>. When all registered threads have called the opration, the LocalCollectiveServer</code> starts the operation. In single node contexts, this is very simple, as the LocalCollectiveServer</code> manages everything with Tensor::to_device</code> for tensor transfers. In a multi-node context, each node will already have the coordinates of other nodes, supplied upon registering. They communicate tensors with the burn_communications</code> crate, specifically with the TensorDataService</code>. This service allows for exposing and downloading Burn tensors over the network in a peer-to-peer manner. Currently we use WebSockets, but QUIC is a likely candidate for future use. In multi-node contexts, nodes must synchronise at the end of the operation. This is true for all collective operations, but it becomes especially important for broadcast</code>, where the broadcaster must wait for all receivers to receive the tensor. Methods</h2> Burn supports multiple strategies for all-reduce, configurable at both local and global levels. Centralized</h3> All peers send tensors to a root, which aggregates them and broadcasts the result back. Tree</h3> Peers are arranged in a b-tree and reduce in parallel, achieving $O(\log_b(N))$ time. Ring</h3> Peers form a ring, slicing tensors and passing them around. This maximizes bandwidth usage but is more sensitive to latency. Figure 4: An overview of the three strategies Local strategy and global strategy</h3> Since the all-reduce is split on two levels, the local (intra-node) level and global (inter-node) level, we can use different local strategies for different nodes, and a different strategy on the global level. Below is a diagram that shows an example of a collective with 3 nodes, each using a different local strategy. Local ring downfall</h3> The Centralized</code> and Tree</code> strategies can be split into two operations: a reduce and a broadcast. A reduce operation aggregates all tensors onto one peer, and a broadcast distributes a tensor from one peer to all others. The result from the global all-reduce needs to be broadcast to all other local peers. So, with Ring</code> and Centralized</code> we don't actually need to do a local all-reduce, we just need to do a reduce, followed by the global all-reduce, followed by the broadcast. It's like fitting the global all-reduce in the middle of the local all-reduce So with Centralized</code> and Tree</code> in multi-node contexts we do: Local reduce -> Global all-reduce -> Local broadcast Due to the nature of the Ring</code> algorithm, a ring-reduce can't be split between a reduce step and a broadcast step. This means if the Ring</code> strategy is chosen locally, the steps will be as follows: Local all-reduce -> Global all-reduce -> Local broadcast This unnecessarily distributes the local all-reduce result to local peers, when anyway we will overwrite the tensor with the global all-reduce result. This may be less advantageous than other configurations. For this reason, it is recommended not to use Ring</code> on the local level, only on the global level. burn-communications</code></h1> With the addition of burn-collective</code>, it was necessary to build a solid base for network communication in burn. The burn-communications</code> crate offers an abstraction of client-server logic, as well as a TensorDataService</code> used for peer-to-peer tensor transfers. This allows developers to swap protocols with minimal effort. DDP Training</h1> Lets get back to a Data Parallel training. How can you take advantage of these fancy new collective operations? Previously, to train on multiple devices, you had to use the LearnerBuilder::devices</code> function: let learner = LearnerBuilder::new(ARTIFACT_DIR) .devices(vec![gpu_1, gpu_2, gpu_3]) // ... .build(model, config.optimizer.init(), 1e-4);</code></pre> This has been replaced with LearnerBuilder::learning_strategy</code>: let collective = CollectiveConfig::default(); let learner = LearnerBuilder::new(ARTIFACT_DIR) .learning_strategy(burn::train::ddp(vec![gpu_1, gpu_2, gpu_3], collective)) // ... .build(model, config.optimizer.init(), 1e-4);</code></pre> The DDP learning strategy will launch a thread for each device, so in single-node environments this is a minimal change. For multi-node environments, the user will need to launch the GlobalOrchestrator</code>. After, they will need to launch the training on each node manually. Extra configuration is also required for the nodes to find each other. let collective = CollectiveConfig::default() .with_global_address(Address::from_str("ws://example.com/orchestrator").unwrap()) .with_num_nodes(3) .with_node_address(Address::from_str("ws://example.com/node_1").unwrap()) .with_data_service_port(3000); let learner = LearnerBuilder::new(ARTIFACT_DIR) .learning_strategy(burn::train::ddp(vec![gpu_1, gpu_2, gpu_3], collective)) // ... .build(model, config.optimizer.init(), 1e-4);</code></pre>Conclusion</h1> With burn-collective and the new DDP learning strategy, training on multiple GPUs or even across multiple nodes is now straightforward in Burn. On a single machine, users only need to provide their devices—the framework handles threading and gradient synchronization automatically. Scaling to multiple nodes requires some extra configuration for the orchestrator and addresses, but the API stays consistent, and the communication layer abstracts away the complexity. The key point is that you don’t need to learn NCCL, MPI, or low-level communication details. Burn provides a unified interface for collective operations that works across devices and nodes, while still letting you choose strategies that match your hardware. This makes it possible to start small and scale up without rewriting your training code. If you’re already training models in Burn, upgrading to distributed data parallel training is just a few lines of code away.

Geþeode - Phonetics Engine

2025-01-01T00:00:00+00:00

Blogs</h2>

Modeling phonology</a></li> </ul>
Example demo</h2>
This demo showcases the `getheode apply</code> tool.`
WASM Geþeode Apply Demo Open fullscreen ↗</a> </div> </iframe> </div> Syntax</h3> Rules follow the notation X → Y / A _ B</code>: change X to Y when preceded by A and followed by B. Example</th> Meaning</th></tr></thead> X -> Y</code></td> change X to Y unconditionally</td></tr> X -> Y / A _ B</code></td> change X to Y when preceded by A, followed by B</td></tr> _</code></td> position of the target in the environment</td></tr> #</code></td> word boundary</td></tr> $</code></td> syllable boundary</td></tr> V</code>, C</code></td> natural classes — vowel, consonant</td></tr> [+voi]</code></td> feature set (any combination of ±</code> features)</td></tr> (X)</code></td> X is optional</td></tr> {a,b,c}</code></td> one of a, b, or c</td></tr> ∅</code> or Ø</code></td> empty segment (deletion or insertion)</td></tr> A B → X Y</code></td> multiple simultaneous rules</td></tr> </tbody></table> Examples</h3> Rule</th> Input</th> Output</th> Description</th></tr></thead> t → d / V_V</code></td> ata</code></td> ada</code></td> intervocalic voicing</td></tr> ə → ∅ / _#</code></td> taɪmə</code></td> taɪm</code></td> word-final schwa deletion</td></tr> θ ð → t d</code></td> ði.θo</code></td> di.to</code></td> multiple rules applied simultaneously</td></tr> {n,q,h} → Ø / _(s)</code></td> oqs.in.ihso</code></td> os.i.iso</code></td> deletion before optional s</td></tr> n → l / #_(V){s,ʃ,h}V{m,b}#</code></td> niham</code></td> liham</code></td> word-initial context with alternatives</td></tr> j → ∅ / Ck_$</code></td> eskj.mo</code></td> esk.mo</code></td> deletion at syllable boundary</td></tr> </tbody></table> Censoring clowns with FPGA 2024-12-03T00:00:00+00:00 In this blog post, I'll talk about an interesting project I worked on, and what I would have done differently. I was working on a demonstration project about face detection and censoring. Instead of designing a CNN/YOLO model on FPGA (which would have been a crazy project on its own), we used simple color detection. The trick: the user would wear a red clown nose, which would be reliable to detect. It was a sort of 'clown censorship device', funny and perfectly acceptable for the context. At first, we used a 256x256 black rectangle to mask the area detected by our red color detection. It was my job to implement a blur in the detection zone. This requires a convolutional filter, with a blurring kernel. This is what the clown censoring looked like. I had it configured to track a blue color, so I was holding up a blue paper on my nose. It was very fun to play with! Game plan</h1> HDMI sweeps the image on the screen in rows: left to right, and top to bottom. So when displaying a pixel, I don't know what color the pixel to the right will be. So to apply a filter on an image, I needed to use some previous frame as input. This meant I needed to store the entire region in memory. My first game plan was as follows: In one frame, store the face's image.</li> On the next frame, display the previous frame's filtered image in the new detection zone.</li> </ul> Downsampling</h2> The first issue I ran into was storing the detected zone in registers. I had access to the pixel colors in RGB888 format, which is 3 bytes per pixel. So I needed $256 \times 256 \times 24 = 1572864$ flip-flops. That was way too much for the simple Zybo Z7-20 FPGA dev-board I was using. I decided to downsample the input by a factor of 16; from 256x256 to 16x16. This inherently added some pixelization to the filter, which was fine. For this crude application, I didn't do any anti-aliasing before downsampling. I used a 3x3 kernel to apply the filter: $$ O(x, y) = \sum_{i=-1}^{1} \sum_{j=-1}^{1} I(x+i, y+j) \times K(i, j) $$ The beautiful thing about FPGA is that all these operations can easily be in parallel. Timing</h2> The second issue was timing. The convolution calculations are hefty, and I needed this done in only a couple clock cycles. This caused timing issues, and I struggled with the screen going black. This is what my code somewhat looked like at this stage: architecture behavioral of conv3x3 is -- 3x3 kernel as float constants constant K00 : float32 := to_float(1.0/16.0, float32'high); constant K01 : float32 := to_float(2.0/16.0, float32'high); constant K02 : float32 := to_float(1.0/16.0, float32'high); constant K10 : float32 := to_float(2.0/16.0, float32'high); constant K11 : float32 := to_float(4.0/16.0, float32'high); constant K12 : float32 := to_float(2.0/16.0, float32'high); constant K20 : float32 := to_float(1.0/16.0, float32'high); constant K21 : float32 := to_float(2.0/16.0, float32'high); constant K22 : float32 := to_float(1.0/16.0, float32'high); signal sum : float32; begin process(all) begin sum := (p00K00) + (p01K01) + (p02K02) + (p10K10) + (p11K11) + (p12K12) + (p20K20) + (p21K21) + (p22*K22); result_out <= sum; end process; end architecture;</code></pre> There were two issues here: I wasn't pipelining my operations</li> I was using floating point arithmetic</li> </ul> I added a 5 stage pipeline that helped divide the operations into smaller more manageable chunks. Now I could assure the whole calculation would only take 5 clock cycles. type FSM_STATE is (IDLE, MULT, ADD_1, ADD_2, ADD_3, OUTPUT);</code></pre> Second, I used integers instead of floats in the calculations. Using integers, I could simply multiply by 1, 2, and 4, and on the output step simply divide by 16 using a right-bit-shift. Using integers allowed me to go much faster, and I saw a noticeable drop in the instability of the design. In the end, we had an architecture that looked something like this: Ultimately, this was a very fun project. Convolutions are everywhere, and opitimizing the calculation for a real-world application is pretty cool. The biggest mistake I made was ignoring BRAM. If I had used BRAM to store the input image, I could have avoided downsampling and taken advantage of the >600 KB of available memory on the Zybo Z7-20 FPGA dev-board. Shattered 2021-06-11T00:00:00+00:00 Shattered by ETN, Kuroyasha, Miya Loustalot, YaBoiCielbird, Valentin Coubronne, Nathan Mercier</a></iframe> Developed in a team of 6 for the GMTK 2021 game jam. Theme: "Joined Together". Ranked #179 overall out of 5,600 submissions top 3%. Built in Unity 3D using C# in 48 hours with all original assets. Out Of Controls 2020-06-12T00:00:00+00:00 Out Of Controls by YaBoiCielbird, TKDR</a></iframe> Developed with a friend for the GMTK 2020 game jam. Theme: "Out of Control". Built in Unity 3D using C#, in 48 hours with all original assets. Outlander's Inn 2020-01-01T00:00:00+00:00 Voxel Planet Demo 2019-08-20T00:00:00+00:00

The largest issue</h2>
Many sound change rules ignore syllable and word boundaries. However, a rule's output cannot be ambiguous, and it cannot be easily inferred. So, this means that some rules are interpreted more strictly than they should, which I'll talk about in a different post.</p>

Centralized</h3>
All peers send tensors to a root, which aggregates them and broadcasts the result back.</p>

Tree</h3>
Peers are arranged in a b-tree and reduce in parallel, achieving $O(\log_b(N))$ time.</p>

Ring</h3>
Peers form a ring, slicing tensors and passing them around. This maximizes bandwidth usage but is more sensitive to latency.</p>
Figure 4: An overview of the three strategies</em></p>

Censoring clowns with FPGA

Shattered

Out Of Controls

Outlander's Inn

Voxel Planet Demo

cielbird

Designing a 12 bit processor

Modeling a conlang's phonology

Writing a Minecraft clone in Rust

Context</h1>
This project started 10 months ago. I wanted to learn more about Rust, and see what it took to write a custom game engine.</p>
And because I grew up with Minecraft, I wanted to shoot my shot at making a Minecraft clone myself.</p>

Distributed AI in Rust

Chickadee - Game Engine

Geþeode - Phonetics Engine

cielbird

Designing a 12 bit processor

Modeling a conlang's phonology

Branching</h2> Getheode also supports optionals or branches in the rule. For example :</p>

Tagging</h2> The tool supports manually tagging features. For example :</p>

The largest issue</h2> Many sound change rules ignore syllable and word boundaries. However, a rule's output cannot be ambiguous, and it cannot be easily inferred. So, this means that some rules are interpreted more strictly than they should, which I'll talk about in a different post.</p>

Writing a Minecraft clone in Rust

Context</h1> This project started 10 months ago. I wanted to learn more about Rust, and see what it took to write a custom game engine.</p> And because I grew up with Minecraft, I wanted to shoot my shot at making a Minecraft clone myself.</p>

Distributed AI in Rust

Collective operations</h1> The syncing of gradients in a data parallel training is a collecive operation called an all-reduce</em>. An all-reduce is one of many primitive collective operations. Some others are:</p>

Centralized</h3> All peers send tensors to a root, which aggregates them and broadcasts the result back.</p>

Tree</h3> Peers are arranged in a b-tree and reduce in parallel, achieving $O(\log_b(N))$ time.</p>

Ring</h3> Peers form a ring, slicing tensors and passing them around. This maximizes bandwidth usage but is more sensitive to latency.</p> Figure 4: An overview of the three strategies</em></p>

Chickadee - Game Engine

Geþeode - Phonetics Engine

Blogs</h2> Modeling phonology</a></li> </ul> Example demo</h2> This demo showcases the getheode apply</code> tool.</p>

Example demo</h2> This demo showcases the getheode apply</code> tool.</p>

Censoring clowns with FPGA

Shattered

Out Of Controls

Outlander's Inn

Voxel Planet Demo

The largest issue</h2>
Many sound change rules ignore syllable and word boundaries. However, a rule's output cannot be ambiguous, and it cannot be easily inferred. So, this means that some rules are interpreted more strictly than they should, which I'll talk about in a different post.</p>

Context</h1>
This project started 10 months ago. I wanted to learn more about Rust, and see what it took to write a custom game engine.</p>
And because I grew up with Minecraft, I wanted to shoot my shot at making a Minecraft clone myself.</p>

Centralized</h3>
All peers send tensors to a root, which aggregates them and broadcasts the result back.</p>

Tree</h3>
Peers are arranged in a b-tree and reduce in parallel, achieving $O(\log_b(N))$ time.</p>

Ring</h3>
Peers form a ring, slicing tensors and passing them around. This maximizes bandwidth usage but is more sensitive to latency.</p>
Figure 4: An overview of the three strategies</em></p>