One of the things I’ve been working on is learning the Unity Entity Component System. This is the central technology underlying DOTS—Data-Oriented Tech Stack—the other of which is the Jobs system.

Jobs are fairly straightforward. They are a Unity-friendly alternative to the .Net Task Parallel library. Tasks are the basis for asynchronous (and concurrent) programming in default .Net.

The asynchronous language features of C# are fine, I guess. It seems like the “async” and “await” features in various languages (like Swift) are pretty similar. (Does Java have something similar now? I have no idea.) But “asynchronous” is not “concurrent”. Asynchrony is a necessary asepct of concurrency, but not at all sufficient.

Asynchrony by itself is the poor developer’s concurrency. It gets you uninterrupted main thread responsiveness, but only because your other tasks are on hold. They are only concurrent insofar as an entirely different processor, on a completely different computer (or other device, or at least a subsytem, like a GPU) is doing the work. Not because you’ve unlocked the power of multiple CPU cores. (Who remembers the days of literally multiple single-core CPUs?)

I first learned concurrency using NSThread, a class that wraps (well, thinly abstracts) POSIX threads in the Foundation framework, part of Mac OS X's Cocoa (formerly NeXTStep and then OpenStep) APIs.

Threads were rarely used by most developers. They weren’t “user friendly” enough, seemingly. They required developers to think about—even understand—arcane concepts of parallel processing, and the perils of sharing data between simultaneous operations.

Not that these constraints have gone away. But modern abstractions enforce good hygiene, in APIs and even languages. Your average developer needs guard rails. (Just like they need automatic memory management.) Library developers must prevent undisciplined coders from hanging themselves with too much rope: that is, to avoid creating race conditions, or non-deterministic behavioural bugs (Heisenbugs), or outright hosing their (customers’) data.

But I digress.

The Unity Job system definitely has training wheels. But you won’t get far if you don’t understand the issues and implications.

Meanwhile, ECS is a essentially a paradigm shift in software design. In some ways, it’s a shift back to the past. Because the past was where developers had to squeeze the most performance they could out of their finite CPU and memory resources.

This has never really changed for games that push the envelope. But fewer games do that. In the past, virtually every game had to push the envelope, or it would be too boring, ugly, and slow to attract many players.

Unity was not initially aimed at envelope-pushing developers. It was aimed at entry- and mid-level developers, who mostly wanted to make games that were pretty similar—at least, technologically—to existing games. It was there to help reduce the coding workload, so more time could be devoted to game design and art direction, or to shorten development time. (It was also a way to sell the dream to wannabe developers, while making money for Unity on the side, through the Unity Asset Store. The Asset Store is the micro-transaction layer of the meta-game that is Unity game development. Most game developers will never ship a finished game, but they might spend a few bucks on 3D models, images, animations, icon sets, shaders, and bootstrapping utilities.)

But something happened: some of Unity’s customers did build and ship some successful games. And those same developers were starting to hit the limits of the GameObject paradigm. (GameObjects are the object-oriented version of entities and components. GameObjects have very little intrinsic game-specific behaviour. You add components—“MonoBehaviours” (historical thing)—as children in a flat object graph. Components can be shared amongst different game objects, and such enhanced game objects, can be serialized as “prefabs”, and re-instantiated as often as necessary.)

ECS blows up the idea of object graphs, and OOP in general, reverting back not only to structures—remember C?—but all the way back to tables (though rotated at ninety degrees, like Google BigTable data). Entities are simply IDs. Components are blocks of data. Components are the input and output of “systems”. Systems are, in essence, old-school C functions. This doesn’t mean that ECS is functional—far from it. Everything is a side effect in ECS, just like it is in SQL or Excel. What use is data if you don’t modify it? Especially in a simulation, where data is a transient resource that serves the instantaneous experiential present. Life is change!

ECS systems run functions that change data willy-nilly. But they do it concurrently, using Jobs.

But what about encapsulation? Sorry. Those classic OOP design principles might ensure data safety, and simplify refactoring, but they also slow things down. How slow? Two orders of magnitude, or worse. And as I said, data safety is irrelevant. We need speed! We need array processing! We need data proximity and cache coherency! And it’s true: these are necessary to ensure that instructions are run smoothly through the pipeline. No branch misprediction! No cache misses!

Making fast code—or making code fast—is not compatible with too many abstractions. Or any abstractions. You have to know how the machine works to squeeze the most out of it. Did I mention that memory management is not compatible with performance? Well I did now.

You can’t use managed objects with ECS and Jobs—not if you want the real performance unlocked by the Burst compiler.

At this point, I have no idea why Unity is still using C#. I suppose there is the syntactic sugar. It might look like you are sending messages to objects, but it’s an illusion provided by C# extensions. (Structs can appear to have methods, but it’s merely a syntax trick of moving the first argument in front of a function name.) Truly, objects themselves have always been illusions. All modern programming constructs are illusions. Human-readable code is an illusion. But some illusions are more costly than others.

Frankly, I think we should go back to C, but with lots of compiler verifications and validations (disabled in release code, except in the no man’s land where untrusted user data is interpreted). So, what was once supposed to be D, and is now either Go or Rust. So, yeah, though I’ve never used it, I kind of wish Unity had adopted Rust, instead of trying to merge two different worlds into the same runtime and language paradigm. You cannot use any of the .Net libraries inside Burst Jobs, so what’s the benefit? But then, webasm is a thing, too.

I think the software development world is completely insane, and on the verge of imploding: collapsing under the weight of its own contradictions. But that’s what you get when the majority of your practitioners don’t even know what an OS kernel is. Or an interrupt. Or a micro-instruction. Or a reference count. But they do know how to delegate their responsibilities to an energy-greedy algorithmic monstrosity that confabulates with total confidence.

So why am I learning ECS? I want to generate meshes procedurally as fast as I can. And just for the hell of it.