Engine Architecture
What Is a Game Engine?
A game engine, in the sense used throughout this page and in Jason Gregory's standard reference on the subject, is the reusable runtime and tooling layer that abstracts away platform details so a team can spend its effort on gameplay rather than re-solving plumbing every project [1].
Engine ≠ Game. The engine is the runtime + tooling; the game is the content + logic that runs on it.
High-Level Architecture
Every general-purpose engine, whatever its marketing name, ends up layered roughly the same way, because the dependencies between the pieces only really work in one direction. The platform abstraction layer sits at the bottom, hiding OS and hardware differences from everything above it; core systems (memory, jobs, assets, logging) sit on top of that, providing the plumbing every runtime system needs; runtime systems (rendering, physics, scripting) build on the core; and editor tooling sits above all of it, calling down into the same runtime the shipped game uses rather than duplicating it. The sections below walk down through these layers roughly in that order.
Vulkan/DX12/Metal] FS[File System] Net[Network] Thread[Threading] end subgraph Core ["Core Systems"] Mem[Memory/Allocators] Job[Job System] Asset[Asset Pipeline] Config[Configuration] Log[Logging/Profiler] end subgraph Runtime ["Runtime Systems"] ECS[ECS / Scene Graph] Physics[Physics] Render[Renderer] AudioSys[Audio Engine] Script[Scripting VM] NetSys[Networking] UI[UI System] end subgraph Tools ["Editor/Tooling"] Editor[Level Editor] AssetTools[Asset Pipeline] Debug[Debug Visualisers] Perf[Profiler UI] Build[Build Pipeline] end %% Connections - using direction to flow down naturally Platform --> Core Core --> Runtime Runtime --> Tools Tools -.-> Asset %% Styling for subgraphs classDef pal fill:#e3f2fd,stroke:#1976d2,stroke-width:2px; classDef core fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px; classDef runtime fill:#e8f5e9,stroke:#388e3c,stroke-width:2px; classDef tools fill:#fff3e0,stroke:#f57c00,stroke-width:2px; class Window,Audio,GPU,FS,Net,Thread pal; class Mem,Job,Asset,Config,Log core; class ECS,Physics,Render,AudioSys,Script,NetSys,UI runtime; class Editor,AssetTools,Debug,Perf,Build tools;
1. Platform Abstraction Layer (PAL)
A game that ships on more than one platform has to contend with the fact that "open a window," "play a sound," and "start a thread" are answered by a completely different API on each one. The platform abstraction layer's whole job is to draw one clean interface — IWindow, an audio backend, a threading primitive — that the rest of the engine codes against, so that everything above it never has to know or care whether it's ultimately talking to Win32, Wayland, or a console SDK.
Why Abstract?
The table below is exactly the kind of divergence that makes a PAL necessary rather than a nice-to-have: not just different APIs for the same concept, but genuinely different concepts (Win32 threads versus pthreads versus GCD) that a single abstraction has to paper over convincingly.
| Platform | Windowing | Graphics | Audio | Threading |
|---|---|---|---|---|
| Windows | Win32 | DX12, Vulkan | XAudio2, WASAPI | Win32 threads |
| Linux | Wayland/X11 | Vulkan | PipeWire/PulseAudio | pthreads |
| macOS | Cocoa | Metal | CoreAudio | pthreads/GCD |
| iOS | UIKit | Metal | AVAudio | pthreads/GCD |
| Android | NativeActivity | Vulkan | OpenSL/AAudio | pthreads |
| Consoles | Proprietary | Proprietary | Proprietary | Proprietary |
PAL Interface Example
Below, IWindow is the abstract interface every platform's window implementation has to satisfy, and a factory function hides which concrete implementation gets constructed behind a single call site. Everything above this layer — input handling, the renderer, the audio system — only ever talks to IWindow, never to a platform-specific window type directly, which is what makes adding a new target platform later a matter of writing one new implementation rather than touching every system that currently opens a window.
# pal/window.py
from dataclasses import dataclass
from abc import ABC, abstractmethod
from typing import Optional
@dataclass
class WindowDesc:
width: int = 1280
height: int = 720
title: str = "Game Window"
fullscreen: bool = False
vsync: bool = True
class IWindow(ABC):
@abstractmethod
def create(self, desc: WindowDesc) -> bool: ...
@abstractmethod
def destroy(self) -> None: ...
@abstractmethod
def poll_events(self) -> None: ...
@abstractmethod
def swap_buffers(self) -> None: ...
@abstractmethod
def get_native_handle(self) -> int: ...
@abstractmethod
def get_size(self) -> tuple[int, int]: ...
@abstractmethod
def set_title(self, title: str) -> None: ...
def create_window(desc: WindowDesc) -> Optional[IWindow]:
"""Factory - implementation per platform"""
pass
# Implementation per platform:
# pal/win32/window.py, pal/linux/window.py, etc.
// pal/window.h
#include <string>
#include <memory>
#include <cstdint>
struct WindowDesc {
uint32_t width = 1280;
uint32_t height = 720;
std::string title = "Game Window";
bool fullscreen = false;
bool vsync = true;
};
class IWindow {
public:
virtual ~IWindow() = default;
virtual bool create(const WindowDesc& desc) = 0;
virtual void destroy() = 0;
virtual void pollEvents() = 0;
virtual void swapBuffers() = 0;
virtual void* getNativeHandle() = 0; // For graphics API
virtual int getWidth() const = 0;
virtual int getHeight() const = 0;
virtual void setTitle(const std::string&) = 0;
};
std::unique_ptr<IWindow> createWindow(const WindowDesc& desc);
// Implementation per platform:
// pal/win32/window.cpp, pal/linux/window.cpp, etc.
// pal/Window.java
package engine.pal;
public interface IWindow {
void create(WindowDesc desc);
void destroy();
void pollEvents();
void swapBuffers();
long getNativeHandle(); // For graphics API
int getWidth();
int getHeight();
void setTitle(String title);
}
public record WindowDesc(
int width,
int height,
String title,
boolean fullscreen,
boolean vsync
) {
public WindowDesc() { this(1280, 720, "Game Window", false, true); }
}
public final class WindowFactory {
public static IWindow createWindow(WindowDesc desc) {
String os = System.getProperty("os.name").toLowerCase();
if (os.contains("win")) return new Win32Window();
if (os.contains("linux")) return new LinuxWindow();
if (os.contains("mac")) return new CocoaWindow();
throw new UnsupportedOperationException("Platform not supported");
}
}
using System;
namespace Engine.PAL {
public interface IWindow {
void Create(WindowDesc desc);
void Destroy();
void PollEvents();
void SwapBuffers();
IntPtr GetNativeHandle(); // For graphics API
int Width { get; }
int Height { get; }
void SetTitle(string title);
}
public record WindowDesc(
uint Width = 1280,
uint Height = 720,
string Title = "Game Window",
bool Fullscreen = false,
bool VSync = true
);
public static class WindowFactory {
public static IWindow CreateWindow(WindowDesc desc) {
var os = Environment.OSVersion.Platform;
if (os == PlatformID.Win32NT) return new Win32Window();
if (os == PlatformID.Unix) return new LinuxWindow();
if (os == PlatformID.MacOSX) return new CocoaWindow();
throw new NotSupportedException("Platform not supported");
}
}
# pal/window.rb
WindowDesc = Struct.new(:width, :height, :title, :fullscreen, :vsync) do
def initialize(width: 1280, height: 720, title: "Game Window",
fullscreen: false, vsync: true)
super(width, height, title, fullscreen, vsync)
end
end
# Interface expressed as a module of required methods
module IWindow
def create(desc) = raise NotImplementedError
def destroy = raise NotImplementedError
def poll_events = raise NotImplementedError
def swap_buffers = raise NotImplementedError
def native_handle = raise NotImplementedError
def size = raise NotImplementedError
def title=(_title)
raise NotImplementedError
end
end
# Factory — implementation per platform
def create_window(desc)
# pal/win32/window.rb, pal/linux/window.rb, etc.
end
2. Memory Management
Reaching for a generic heap allocator (malloc/new) on every allocation is fine for most software, but a game running at 60 frames per second has roughly sixteen milliseconds to do everything for that frame, and a general-purpose allocator's bookkeeping — searching free lists, taking locks, risking fragmentation — eats into a budget that has no room to spare. Engines instead reach for a small toolbox of specialised allocators, each shaped around one specific allocation lifetime, trading the generality of malloc for predictable, close-to-free performance in the case that actually matters.
Allocator Strategy
The four allocators below cover almost every lifetime pattern a game needs. A linear allocator just walks a pointer forward through a pre-allocated buffer and resets it to the start at a frame boundary — freeing is instantaneous because nothing is ever freed individually, the whole arena is reclaimed at once. A pool allocator does the same trick for fixed-size objects that come and go constantly (entities, particles), keeping a free list of slots rather than a moving pointer. A stack allocator adds scoped, nested lifetimes on top of a linear allocator via lightweight markers. And a tracking allocator wraps any of the above with bookkeeping — deliberately slower, and used only in debug builds, purely to catch leaks before they reach a shipped build.
# Linear allocator — frame/temporary allocations
class LinearAllocator:
def __init__(self, size: int):
self.size = size
self.offset = 0
self.buffer = bytearray(size)
def allocate(self, size: int, alignment: int = 16) -> memoryview:
self.offset = (self.offset + alignment - 1) & ~(alignment - 1)
assert self.offset + size <= self.size, "Out of memory"
ptr = memoryview(self.buffer)[self.offset:self.offset + size]
self.offset += size
return ptr
def reset(self) -> None:
self.offset = 0 # Frame boundary
# Pool allocator — fixed-size objects (entities, components)
from typing import TypeVar, Generic, List
T = TypeVar('T')
class PoolAllocator(Generic[T]):
def __init__(self, factory, initial_size: int = 100):
self.factory = factory
self.pool: List[T] = [factory() for _ in range(initial_size)]
self.free_list: List[int] = list(range(initial_size))
def allocate(self) -> T:
if self.free_list:
idx = self.free_list.pop()
return self.pool[idx]
self.pool.append(self.factory())
return self.pool[-1]
def deallocate(self, obj: T) -> None:
idx = self.pool.index(obj)
self.free_list.append(idx)
# Stack allocator — scoped allocations with automatic cleanup
class StackAllocator:
def __init__(self, size: int):
self.base = LinearAllocator(size)
class Marker:
def __init__(self, offset: int):
self.offset = offset
def get_marker(self) -> Marker:
return self.base.Marker(self.base.offset)
def free_to_marker(self, marker: Marker) -> None:
self.base.offset = marker.offset
# Standard allocator with tracking (debug)
class TrackingAllocator:
def __init__(self):
self.allocations = {}
import threading
self.mutex = threading.Lock()
def allocate(self, size: int, file: str, line: int) -> memoryview:
with self.mutex:
ptr = bytearray(size)
self.allocations[id(ptr)] = (size, file, line)
return memoryview(ptr)
def deallocate(self, ptr: memoryview) -> None:
with self.mutex:
if id(ptr) in self.allocations:
del self.allocations[id(ptr)]
def report_leaks(self) -> None:
for ptr, (size, file, line) in self.allocations.items():
print(f"LEAK: {size} bytes at {file}:{line}")
// Linear allocator — frame/temporary allocations
class LinearAllocator {
char* buffer;
size_t offset, size;
public:
LinearAllocator(size_t size) : size(size), offset(0) {
buffer = static_cast<char*>(malloc(size));
}
void* allocate(size_t size, size_t alignment = 16) {
offset = (offset + alignment - 1) & ~(alignment - 1);
assert(offset + size <= this->size);
void* ptr = buffer + offset;
offset += size;
return ptr;
}
void reset() { offset = 0; } // Frame boundary
~LinearAllocator() { free(buffer); }
};
// Pool allocator — fixed-size objects (entities, components)
template<typename T>
class PoolAllocator {
std::vector<T> pool;
std::vector<size_t> freeList;
public:
explicit PoolAllocator(size_t initial = 100) {
pool.reserve(initial);
for (size_t i = 0; i < initial; ++i) {
pool.emplace_back();
freeList.push_back(i);
}
}
template<typename... Args>
T* allocate(Args&&... args) {
T* obj;
if (!freeList.empty()) {
obj = &pool[freeList.back()];
freeList.pop_back();
new (obj) T(); // Placement new
} else {
pool.emplace_back();
obj = &pool.back();
}
return obj;
}
void deallocate(T* ptr) {
ptr->~T();
freeList.push_back(ptr - pool.data());
}
};
// Stack allocator — scoped allocations with automatic cleanup
class StackAllocator {
struct Marker { size_t offset; };
LinearAllocator base;
public:
Marker getMarker() { return {base.offset}; }
void freeToMarker(Marker m) { base.offset = m.offset; }
};
// Standard allocator with tracking (debug)
class TrackingAllocator {
std::unordered_map<void*, AllocationInfo> allocations;
std::mutex mutex;
public:
void* allocate(size_t size, const char* file, int line) {
std::lock_guard lock(mutex);
void* ptr = malloc(size);
allocations[ptr] = {size, file, line};
return ptr;
}
void deallocate(void* ptr) {
std::lock_guard lock(mutex);
allocations.erase(ptr);
free(ptr);
}
void reportLeaks() const {
for (auto& [ptr, info] : allocations)
std::cout << "LEAK: " << info.size << "B at " << info.file << ":" << info.line << "\n";
}
};
import java.util.*;
import java.util.concurrent.locks.*;
public final class LinearAllocator {
private final byte[] buffer;
private int offset, size;
public LinearAllocator(int size) {
this.size = size;
this.buffer = new byte[size];
}
public ByteBuffer allocate(int size, int alignment) {
offset = (offset + alignment - 1) & ~(alignment - 1);
if (offset + size > this.size) throw new OutOfMemoryError();
int pos = offset;
offset += size;
return ByteBuffer.wrap(buffer, pos, size).asReadOnlyBuffer();
}
public void reset() { offset = 0; }
}
public final class PoolAllocator<T> {
private final Supplier<T> factory;
private final List<T> pool = new ArrayList<>();
private final Deque<Integer> freeList = new ArrayDeque<>();
public PoolAllocator(Supplier<T> factory, int initial) {
this.factory = factory;
for (int i = 0; i < initial; i++) {
pool.add(factory.get());
freeList.add(i);
}
}
public T allocate() {
if (!freeList.isEmpty()) return pool.get(freeList.pop());
T obj = factory.get();
pool.add(obj);
return obj;
}
public void deallocate(T obj) {
int idx = pool.indexOf(obj);
if (idx >= 0) freeList.add(idx);
}
}
public final class StackAllocator {
private final LinearAllocator base = new LinearAllocator(0);
public record Marker(int offset) {}
public Marker getMarker() { return new Marker(offset); }
public void freeToMarker(Marker m) { offset = m.offset; }
}
public final class TrackingAllocator {
private final Map<Long, AllocationInfo> allocations = new ConcurrentHashMap<>();
public ByteBuffer allocate(int size, String file, int line) {
ByteBuffer buf = ByteBuffer.allocateDirect(size);
allocations.put((long) buf.hashCode(), new AllocationInfo(size, file, line));
return buf;
}
public void deallocate(ByteBuffer buf) {
allocations.remove((long) buf.hashCode());
}
public void reportLeaks() {
allocations.forEach((k, v) ->
System.out.println("LEAK: " + v.size + "B at " + v.file + ":" + v.line));
}
private record AllocationInfo(int size, String file, int line) {}
}
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
// Linear allocator — frame/temporary allocations
public unsafe class LinearAllocator {
private byte* buffer;
private int offset, size;
public LinearAllocator(int size) {
this.size = size;
this.buffer = (byte*) Marshal.AllocHGlobal(size);
}
public Span<byte> Allocate(int size, int alignment = 16) {
offset = (offset + alignment - 1) & ~(alignment - 1);
if (offset + size > this.size) throw new OutOfMemoryException();
int pos = offset;
offset += size;
return new Span<byte>(buffer + pos, size);
}
public void Reset() => offset = 0;
public void Dispose() {
if (buffer != null) Marshal.FreeHGlobal((IntPtr)buffer);
buffer = null;
}
}
// Pool allocator — fixed-size objects (entities, components)
public class PoolAllocator<T> where T : new() {
private readonly List<T> pool = new();
private readonly Stack<int> freeList = new();
public PoolAllocator(int initial = 100) {
for (int i = 0; i < initial; i++) {
pool.Add(new T());
freeList.Push(i);
}
}
public T Allocate() {
if (freeList.Count > 0) return pool[freeList.Pop()];
var obj = new T();
pool.Add(obj);
pool.Add(obj);
return obj;
}
public void Deallocate(T obj) {
int idx = pool.IndexOf(obj);
if (idx >= 0) freeList.Push(idx);
}
}
// Stack allocator — scoped allocations with automatic cleanup
public class StackAllocator {
private readonly LinearAllocator baseAlloc = new();
public readonly struct Marker { public readonly int Offset; }
public Marker GetMarker() => new() { Offset = baseAlloc.Offset };
public void FreeToMarker(Marker m) => baseAlloc.Offset = m.Offset;
}
// Standard allocator with tracking (debug)
public class TrackingAllocator {
private readonly Dictionary<IntPtr, AllocationInfo> allocations = new();
private readonly Mutex mutex = new();
public IntPtr Allocate(int size, string file, int line) {
lock (mutex) {
IntPtr ptr = Marshal.AllocHGlobal(size);
allocations[ptr] = new AllocationInfo(size, file, line);
return ptr;
}
}
public void Deallocate(IntPtr ptr) {
lock (mutex) {
allocations.Remove(ptr);
Marshal.FreeHGlobal(ptr);
}
}
public void ReportLeaks() {
foreach (var (ptr, info) in allocations)
Console.WriteLine($"LEAK: {info.Size}B at {info.File}:{info.Line}");
}
public readonly record struct AllocationInfo(int Size, string File, int Line);
}
# Linear allocator — frame/temporary allocations
class LinearAllocator
attr_accessor :offset
def initialize(size)
@size = size
@offset = 0
@buffer = String.new("\0" * size, encoding: Encoding::BINARY)
end
def allocate(size, alignment = 16)
@offset = (@offset + alignment - 1) & ~(alignment - 1)
raise "Out of memory" if @offset + size > @size
slice_start = @offset
@offset += size
@buffer[slice_start, size] # byteslice of the arena
end
def reset
@offset = 0 # Frame boundary
end
end
# Pool allocator — fixed-size objects (entities, components)
class PoolAllocator
def initialize(initial_size = 100, &factory)
@factory = factory
@pool = Array.new(initial_size) { factory.call }
@free_list = (0...initial_size).to_a
end
def allocate
if (idx = @free_list.pop)
@pool[idx]
else
@pool << @factory.call
@pool.last
end
end
def deallocate(obj)
idx = @pool.index(obj)
@free_list << idx if idx
end
end
# Stack allocator — scoped allocations with automatic cleanup
class StackAllocator
Marker = Struct.new(:offset)
def initialize(size)
@base = LinearAllocator.new(size)
end
def marker
Marker.new(@base.offset)
end
def free_to_marker(marker)
@base.offset = marker.offset
end
end
# Standard allocator with tracking (debug)
class TrackingAllocator
def initialize
@allocations = {}
@mutex = Mutex.new
end
def allocate(size, file, line)
@mutex.synchronize do
ptr = String.new("\0" * size, encoding: Encoding::BINARY)
@allocations[ptr.object_id] = [size, file, line]
ptr
end
end
def deallocate(ptr)
@mutex.synchronize { @allocations.delete(ptr.object_id) }
end
def report_leaks
@allocations.each_value do |size, file, line|
puts "LEAK: #{size} bytes at #{file}:#{line}"
end
end
end
Allocation Rules
Picking the right allocator for a given piece of data is mostly a question of how long that data needs to live, which is why the table below is organised by lifetime rather than by data type — the same Vec3 might belong in a frame-linear buffer one moment and a long-lived pool the next, depending entirely on what's holding onto it.
| Allocator | Use For | Lifetime |
|---|---|---|
| System heap | Engine subsystems, long-lived | Process |
| Frame linear | Per-frame render data, temp maths | Frame |
| Pool | Entities, components, particles | Object lifetime |
| Stack | Scoped function allocations | Scope |
| Tracking | Debug builds only | N/A |
3. Job System (Parallelism)
Modern hardware offers far more CPU cores than a single-threaded frame loop can use, but games have historically been reluctant to spread work across threads by hand — shared mutable state between systems makes ad-hoc threading a reliable source of hard-to-reproduce bugs. A job system is the engine's answer: instead of manually spinning up threads for specific tasks, code submits small, independent units of work ("jobs") to a shared pool, optionally declaring dependencies between them, and lets a scheduler handle when and where each one actually runs.
Why Job System?
Games need frame-parallelism (render + sim + audio) and data-parallelism (culling, skinning, particles). The dependency-aware handle below is what makes both viable safely: a job can declare it must wait on others before running, so the scheduler can freely reorder and parallelise everything that has no such dependency, while still guaranteeing that (for example) rendering never reads a physics result computed on another thread before that computation has actually finished.
import threading
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass
from typing import Callable, Optional
import concurrent.futures
@dataclass
class JobHandle:
counter: int = 1
class JobSystem:
def __init__(self, num_threads: int = None):
self.workers = ThreadPoolExecutor(max_workers=num_threads or threading.cpu_count())
self._queue = concurrent.futures.Queue()
self._running = True
# Fire-and-forget
def dispatch(self, func: Callable[[], None]) -> JobHandle:
handle = JobHandle()
self._queue.put((func, handle))
return handle
# With dependency
def dispatch_with_deps(self, func: Callable[[], None], deps: list[JobHandle]) -> JobHandle:
h = JobHandle(1 + len(deps))
for d in deps:
d.counter -= 1
self._queue.put((func, h))
return h
def wait(self, handle: JobHandle) -> None:
while handle.counter > 0:
self._execute_one()
def wait_all(self) -> None:
pass # drain queue
def _execute_one(self) -> None:
try:
func, handle = self._queue.get_nowait()
if handle.counter == 0:
func()
else:
self._queue.put((func, handle))
except Exception:
pass
#include <functional>
#include <vector>
#include <thread>
#include <atomic>
#include <queue>
#include <mutex>
class JobSystem {
struct Job {
std::function<void()> work;
std::atomic<int>* counter;
};
struct Handle { std::atomic<int> counter{1}; };
std::vector<std::thread> workers;
std::queue<Job> queue;
std::mutex queue_mutex;
std::atomic<bool> running{true};
JobSystem(int numThreads = std::thread::hardware_concurrency());
// Fire-and-forget
template<typename F>
void dispatch(F&& f) {
std::lock_guard lock(queue_mutex);
queue.push({std::forward<F>(f), nullptr});
}
// With dependency
template<typename F>
Handle dispatch(F&& f, Handle* deps, int numDeps) {
Handle h;
h.counter.store(1 + numDeps);
for (int i = 0; i < numDeps; ++i) deps[i].counter.fetch_sub(1);
std::lock_guard lock(queue_mutex);
queue.push({std::forward<F>(f), &h.counter});
return h;
}
void wait(const Handle& h) {
while (h.counter.load() > 0) executeOne();
}
void waitAll() { /* drain queue */ }
private:
void executeOne();
};
import java.util.concurrent.*;
import java.util.function.*;
import java.util.*;
public final class JobSystem {
private record Job(Runnable work, AtomicInteger counter) {}
public static class Handle {
public final AtomicInteger counter = new AtomicInteger(1);
}
private final ExecutorService executor;
private final ConcurrentLinkedQueue<Job> queue = new ConcurrentLinkedQueue<>();
private final AtomicBoolean running = new AtomicBoolean(true);
public JobSystem(int threads) {
executor = Executors.newFixedThreadPool(threads);
}
public void dispatch(Runnable work) {
queue.add(new Job(work, null));
}
public Handle dispatch(Runnable work, Handle... deps) {
Handle h = new Handle();
h.counter.set(1 + deps.length);
for (Handle d : deps) d.counter.decrementAndGet();
queue.add(new Job(work, h.counter));
return h;
}
public void wait(Handle h) {
while (h.counter.get() > 0) executeOne();
}
public void waitAll() { /* drain queue */ }
private void executeOne() {
Job job = queue.poll();
if (job != null && (job.counter == null || job.counter.get() == 0))
job.work.run();
else if (job != null) queue.add(job);
}
}
using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;
public class JobSystem {
public class Handle {
public int Counter = 1;
}
private readonly ConcurrentQueue<(Action Work, Handle Counter)> _queue = new();
private readonly Thread[] _workers;
private readonly CancellationTokenSource _cts = new();
public JobSystem(int threads = -1) {
threads = threads > 0 ? threads : Environment.ProcessorCount;
_workers = new Thread[threads];
for (int i = 0; i < threads; i++) {
_workers[i] = new Thread(ExecuteLoop) { IsBackground = true };
_workers[i].Start();
}
}
public void Dispatch(Action work) {
_queue.Enqueue((work, null));
}
public Handle Dispatch(Action work, params Handle[] deps) {
var h = new Handle { Counter = 1 + deps.Length };
foreach (var d in deps) Interlocked.Decrement(ref d.Counter);
_queue.Enqueue((work, h));
return h;
}
public void Wait(Handle h) {
while (Volatile.Read(ref h.Counter) > 0) ExecuteOne();
}
public void WaitAll() { }
private void ExecuteLoop() {
while (!_cts.IsCancellationRequested) {
ExecuteOne();
}
}
private void ExecuteOne() {
if (_queue.TryDequeue(out var job)) {
if (job.Counter == null || job.Counter.Counter == 0)
job.Work();
else
_queue.Enqueue(job);
}
}
}
JobHandle = Struct.new(:counter) do
def initialize(counter = 1) = super
end
class JobSystem
def initialize(num_threads = nil)
@queue = Queue.new
@running = true
count = num_threads || Etc.nprocessors
@workers = Array.new(count) do
Thread.new { execute_one while @running }
end
end
# Fire-and-forget
def dispatch(&func)
handle = JobHandle.new
@queue << [func, handle]
handle
end
# With dependency
def dispatch_with_deps(deps, &func)
handle = JobHandle.new(1 + deps.size)
deps.each { |d| d.counter -= 1 }
@queue << [func, handle]
handle
end
def wait(handle)
execute_one while handle.counter > 0
end
def wait_all
# drain queue
end
private
def execute_one
func, handle = @queue.pop(true)
if handle.counter.zero?
func.call
else
@queue << [func, handle]
end
rescue ThreadError
nil # queue empty
end
end
Frame Pipeline with Jobs
Put together, a real frame looks like the sketch below: input stays single-threaded because it has to happen first and is cheap anyway, simulation and culling are dispatched in parallel because neither depends on the other, rendering waits only on the specific job it actually needs (updated positions), and audio runs independently on its own thread throughout. Notice what the job system buys here beyond raw parallelism: the tick function reads as a straightforward, still-sequential-looking description of "what happens this frame," while the actual thread scheduling is handled entirely by the dependency handles underneath it.
def tick(dt: float) -> None:
# 1. Input (main thread)
input.poll()
# 2. Parallel: Simulation + Render prep
sim_handle = jobs.dispatch(lambda: (physics.step(dt), ecs.update(dt)))
cull_handle = jobs.dispatch(lambda: (frustum_cull(camera), occlusion_cull()))
# 3. Wait for sim done (render needs positions)
jobs.wait(sim_handle)
# 4. Render command generation (main or render thread)
render.generate_commands()
# 5. Audio (separate thread)
audio.update()
# 6. Present
window.swap_buffers()
# 6.5 Frame cleanup
frame_allocator.reset()
void Game::tick(float dt) {
// 1. Input (main thread)
input.poll();
// 2. Parallel: Simulation + Render prep
JobHandle simHandle = jobs.dispatch([&] {
physics.step(dt);
ecs.update(dt); // Systems in parallel via archetypes
});
JobHandle cullHandle = jobs.dispatch([&] {
frustumCull(camera);
occlusionCull();
});
// 3. Wait for sim done (render needs positions)
jobs.wait(simHandle);
// 4. Render command generation (main or render thread)
render.generateCommands();
// 5. Audio (separate thread)
audio.update();
// 6. Present
window.swapBuffers();
// 6.5 Frame cleanup
frameAllocator.reset();
}
public void tick(float dt) {
// 1. Input (main thread)
input.poll();
// 2. Parallel: Simulation + Render prep
JobSystem.Handle simHandle = jobs.dispatch(() -> {
physics.step(dt);
ecs.update(dt);
});
JobSystem.Handle cullHandle = jobs.dispatch(() -> {
frustumCull(camera);
occlusionCull();
});
// 3. Wait for sim done (render needs positions)
jobs.wait(simHandle);
// 4. Render command generation (main or render thread)
render.generateCommands();
// 5. Audio (separate thread)
audio.update();
// 6. Present
window.swapBuffers();
// 6.5 Frame cleanup
frameAllocator.reset();
}
public void Tick(float dt) {
// 1. Input (main thread)
_input.Poll();
// 2. Parallel: Simulation + Render prep
var simHandle = _jobs.Dispatch(() => {
_physics.Step(dt);
_ecs.Update(dt);
});
var cullHandle = _jobs.Dispatch(() => {
FrustumCull(_camera);
OcclusionCull();
});
// 3. Wait for sim done (render needs positions)
_jobs.Wait(simHandle);
// 4. Render command generation (main or render thread)
_render.GenerateCommands();
// 5. Audio (separate thread)
_audio.Update();
// 6. Present
_window.SwapBuffers();
// 6.5 Frame cleanup
_frameAllocator.Reset();
}
def tick(dt)
# 1. Input (main thread)
input.poll
# 2. Parallel: Simulation + Render prep
sim_handle = jobs.dispatch do
physics.step(dt)
ecs.update(dt)
end
cull_handle = jobs.dispatch do
frustum_cull(camera)
occlusion_cull
end
# 3. Wait for sim done (render needs positions)
jobs.wait(sim_handle)
# 4. Render command generation (main or render thread)
render.generate_commands
# 5. Audio (separate thread)
audio.update
# 6. Present
window.swap_buffers
# 6.5 Frame cleanup
frame_allocator.reset
end
4. Asset Pipeline
Almost nothing that ends up on screen or in the game's audio was authored in the format the engine actually consumes at runtime — an artist's FBX model or a composer's WAV file has to pass through several transformations before it's something the GPU or audio hardware can use efficiently. The asset pipeline is the machinery that performs those transformations once, ahead of time, rather than repeating expensive work (like re-parsing a source model file) every time the game loads.
Build-Time vs Runtime
Splitting this work into distinct phases means the expensive parts — parsing exotic source formats, compressing textures, baking lighting — happen once on a developer's machine or a build server, leaving the runtime path as fast as possible: stream a pre-processed, pre-packaged file and get it onto the GPU with minimal further work.
| Phase | Operations |
|---|---|
| Import | Source (FBX, PSD, WAV) → Engine intermediate |
| Process | Optimise, compress, generate mipmaps, bake |
| Package | Bundle into platform-specific archives |
| Runtime Load | Stream, decompress, GPU upload |
Intermediate Formats
The import step's output is an engine-defined intermediate representation, deliberately stripped down to only what the runtime actually needs — a mesh keeps its vertices, indices and bounds but not whatever editor-only metadata the source DCC tool attached; a texture keeps compressed pixel data and format flags but nothing about how it was authored. Designing this intermediate format well is most of the asset pipeline's real engineering effort, since every importer for every source format has to converge on it.
from dataclasses import dataclass
from typing import List, Dict
@dataclass
class Vertex:
position: tuple[float, float, float]
normal: tuple[float, float, float]
uv: tuple[float, float]
tangent: tuple[float, float, float, float]
@dataclass
class MeshAsset:
vertices: List[Vertex]
indices: List[int] # 16 or 32-bit
submeshes: List['Submesh']
bounds: 'Bounds'
skeleton: Optional['Skeleton'] = None
blend_shapes: List['BlendShape'] = []
@dataclass
class TextureAsset:
width: int
height: int
depth: int
mip_levels: int
format: str # BC7, ASTC, etc.
data: bytes # Compressed
is_srgb: bool
is_normal_map: bool
@dataclass
class MaterialAsset:
shader: str # Shader permutation key
textures: Dict[str, 'TextureHandle']
scalars: Dict[str, float]
vectors: Dict[str, tuple[float, float, float, float]]
state: 'RenderState'
// Mesh intermediate (after import, before GPU)
struct Vertex {
Vec3 position, normal, tangent;
Vec2 uv;
};
struct MeshAsset {
std::vector<Vertex> vertices; // Position, normal, UV, tangent
std::vector<uint32_t> indices; // 16 or 32-bit
std::vector<Submesh> submeshes; // Material ranges
Bounds bounds; // AABB, sphere
Skeleton skeleton; // Optional
std::vector<BlendShape> shapes; // Optional
};
// Texture intermediate
struct TextureAsset {
uint32_t width, height, depth;
uint32_t mipLevels;
Format format; // BC7, ASTC, etc.
std::vector<uint8_t> data; // Compressed
bool isSRGB;
bool isNormalMap;
};
// Material intermediate
struct MaterialAsset {
std::string shader; // Shader permutation key
std::unordered_map<std::string, TextureHandle> textures;
std::unordered_map<std::string, float> scalars;
std::unordered_map<std::string, Vec4> vectors;
RenderState state; // Blend, depth, cull
};
import java.util.*;
import java.util.function.*;
public final class Vertex {
public final Vec3 position, normal, tangent;
public final Vec2 uv;
public Vertex(Vec3 pos, Vec3 norm, Vec3 tan, Vec2 uv) {
this.position = pos; this.normal = norm; this.tangent = tan; this.uv = uv;
}
}
public final class MeshAsset {
public final List<Vertex> vertices; // Position, normal, UV, tangent
public final List<Integer> indices; // 16 or 32-bit
public final List<Submesh> submeshes; // Material ranges
public final Bounds bounds; // AABB, sphere
public final Skeleton skeleton; // Optional
public final List<BlendShape> shapes; // Optional
}
public final class TextureAsset {
public final int width, height, depth;
public final int mipLevels;
public final Format format; // BC7, ASTC, etc.
public final byte[] data; // Compressed
public final boolean isSRGB;
public final boolean isNormalMap;
}
public final class MaterialAsset {
public final String shader; // Shader permutation key
public final Map<String, TextureHandle> textures;
public final Map<String, Float> scalars;
public final Map<String, Vec4> vectors;
public final RenderState state; // Blend, depth, cull
}
// Mesh intermediate (after import, before GPU)
public readonly record struct Vertex(
Vec3 Position, Vec3 Normal, Vec3 Tangent, Vec2 UV
);
public sealed class MeshAsset {
public List<Vertex> Vertices { get; } = new();
public List<int> Indices { get; } = new(); // 16 or 32-bit
public List<Submesh> Submeshes { get; } = new();
public Bounds Bounds { get; init; }
public Skeleton Skeleton { get; init; }
public List<BlendShape> Shapes { get; } = new();
}
public sealed class TextureAsset {
public uint Width, Height, Depth;
public uint MipLevels;
public Format Format; // BC7, ASTC, etc.
public byte[] Data; // Compressed
public bool IsSRGB;
public bool IsNormalMap;
}
public sealed class MaterialAsset {
public string Shader; // Shader permutation key
public Dictionary<string, TextureHandle> Textures { get; } = new();
public Dictionary<string, float> Scalars { get; } = new();
public Dictionary<string, Vec4> Vectors { get; } = new();
public RenderState State { get; init; }
}
Vertex = Struct.new(:position, :normal, :uv, :tangent)
MeshAsset = Struct.new(:vertices, :indices, :submeshes, :bounds,
:skeleton, :blend_shapes) do
def initialize(vertices:, indices:, submeshes:, bounds:,
skeleton: nil, blend_shapes: [])
super(vertices, indices, submeshes, bounds, skeleton, blend_shapes)
end
end
TextureAsset = Struct.new(
:width, :height, :depth, :mip_levels,
:format, # BC7, ASTC, etc.
:data, # Compressed bytes
:is_srgb, :is_normal_map
)
MaterialAsset = Struct.new(
:shader, # Shader permutation key
:textures, # { name => TextureHandle }
:scalars, # { name => Float }
:vectors, # { name => [x, y, z, w] }
:state # RenderState
)
Asset Registry (Runtime)
At runtime, an asset registry sits between game code asking for an asset by ID and the actual bytes sitting on disk or in a package, handling loading, caching, and — critically for anything running on constrained hardware — eviction. Weak references let an asset be garbage-collected once nothing is using it any more, and an LRU-style budget keeps the working set within whatever memory the platform actually has, evicting the least-recently-used entries first when a new load would push it over.
from weakref import WeakValueDictionary
from typing import TypeVar, Generic
T = TypeVar('T')
class AssetRegistry:
def __init__(self):
self._assets = {}
self._path_to_id = {}
self._gpu_cache = LRUCache()
def load(self, asset_id: T) -> T:
entry = self._assets[asset_id]
if ptr := entry.data():
entry.last_used_frame = frame
return ptr
# Load from disk → process → upload GPU
data = load_from_package(asset_id)
entry.data = data
entry.last_used_frame = frame
return data
def update(self, budget_bytes: int) -> None:
# Evict LRU entries over budget
while self.current_size > budget_bytes:
evict_oldest()
class AssetRegistry {
struct AssetEntry {
AssetID id;
std::string path;
std::weak_ptr<void> data; // Auto-unload if unused
uint64_t lastUsedFrame;
size_t sizeBytes;
};
std::unordered_map<AssetID, AssetEntry> assets;
std::unordered_map<std::string, AssetID> pathToID;
LRUCache<AssetID, std::shared_ptr<void>> gpuCache;
public:
template<typename T>
std::shared_ptr<T> load(AssetID id) {
auto& entry = assets[id];
if (auto ptr = entry.data.lock()) {
entry.lastUsedFrame = frame;
return std::static_pointer_cast<T>(ptr);
}
// Load from disk → process → upload GPU
auto data = loadFromPackage<T>(id);
entry.data = data;
entry.lastUsedFrame = frame;
return data;
}
void update(size_t budgetBytes) {
// Evict LRU entries over budget
while (currentSize > budgetBytes) evictOldest();
}
};
public final class AssetRegistry {
private final Map<AssetID, AssetEntry> assets = new ConcurrentHashMap<>();
private final Map<String, AssetID> pathToID = new ConcurrentHashMap<>();
private final LRUCache<AssetID, Object> gpuCache = new LRUCache<>();
public record AssetEntry(
AssetID id, String path,
WeakReference<Object> data,
long lastUsedFrame, long sizeBytes
) {}
@SuppressWarnings("unchecked")
public <T> T load(AssetID id) {
AssetEntry entry = assets.get(id);
if (entry != null && entry.data.get() != null) {
entry.lastUsedFrame = frame;
return (T) entry.data.get();
}
T data = loadFromPackage(id);
entry.data = new WeakReference<>(data);
entry.lastUsedFrame = frame;
return data;
}
public void update(long budgetBytes) {
while (currentSize > budgetBytes) evictOldest();
}
}
public class AssetRegistry {
private readonly Dictionary<AssetID, AssetEntry> _assets = new();
private readonly Dictionary<string, AssetID> _pathToId = new();
private readonly LRUCache<AssetID, object> _gpuCache = new();
private record AssetEntry(
AssetID Id, string Path,
WeakReference Data,
long LastUsedFrame, long SizeBytes
);
public T Load<T>(AssetID id) where T : class {
if (_assets.TryGetValue(id, out var entry) && entry.Data.IsAlive) {
entry.LastUsedFrame = _frame;
return (T)entry.Data.Target!;
}
var data = LoadFromPackage<T>(id);
_assets[id] = new AssetEntry(id, "", new WeakReference(data), _frame, 0);
return data;
}
public void Update(long budgetBytes) {
while (CurrentSize > budgetBytes) EvictOldest();
}
}
class AssetRegistry
def initialize
@assets = {}
@path_to_id = {}
@gpu_cache = LRUCache.new
end
def load(asset_id)
entry = @assets[asset_id]
if (ptr = entry.data)
entry.last_used_frame = frame
return ptr
end
# Load from disk → process → upload GPU
data = load_from_package(asset_id)
entry.data = data
entry.last_used_frame = frame
data
end
def update(budget_bytes)
# Evict LRU entries over budget
evict_oldest while current_size > budget_bytes
end
end
5. Renderer Architecture
A modern frame is rendered in dozens of distinct passes — shadow maps, a G-buffer, lighting, post-processing — each reading some GPU resources and writing others, with a web of dependencies between them that only grows as more passes are added. Hand-managing that dependency graph (deciding what order passes run in, which resources can share memory, where a barrier or synchronisation point is needed) becomes unmanageable by hand once a renderer has more than a handful of passes, which is what has pushed most modern engines toward describing the frame declaratively instead.
Render Graph (Modern)
A render graph lets each pass simply declare what it reads and writes, and leaves the actual scheduling, resource aliasing, and barrier placement to a compiler step that runs once the whole graph is known. This is the single biggest reason render graphs caught on industry-wide: a pass author never has to reason globally about the whole frame, only locally about their own inputs and outputs, while the compiler still produces a globally efficient schedule.
from typing import Callable, List
from dataclasses import dataclass
@dataclass
class ResourceRef:
name: str
@dataclass
class Pass:
name: str
execute: Callable[['RenderContext'], None]
reads: List[ResourceRef] = field(default_factory=list)
writes: List[ResourceRef] = field(default_factory=list)
class RenderGraph:
def __init__(self):
self.passes: List[Pass] = []
self.resources = ResourcePool() # Transient (RTs, buffers)
def add_pass(self, name: str, execute: Callable[['RenderContext'], None]) -> 'RenderGraph':
self.passes.append(Pass(name, execute))
return self
def reads(self, ref: ResourceRef) -> 'RenderGraph':
self.passes[-1].reads.append(ref)
return self
def writes(self, ref: ResourceRef) -> 'RenderGraph':
self.passes[-1].writes.append(ref)
return self
def compile(self) -> None:
pass # Topological sort, alias transient resources
def execute(self, ctx: 'RenderContext') -> None:
for pass_ in self.passes:
pass_.execute(ctx)
# Usage per frame
rg = RenderGraph()
rg.add_pass("ShadowMap", lambda c: render_shadows(c)) \
.writes(ResourceRef("shadowMap")) \
.reads(ResourceRef("sceneDepth"))
rg.add_pass("GBuffer", lambda c: render_gbuffer(c)) \
.writes(ResourceRef("gbuffer0"), ResourceRef("gbuffer1"),
ResourceRef("gbuffer2"), ResourceRef("depth")) \
.reads(ResourceRef("shadowMap"))
rg.add_pass("Lighting", lambda c: render_deferred(c)) \
.writes(ResourceRef("hdrColor")) \
.reads(ResourceRef("gbuffer0"), ResourceRef("gbuffer1"),
ResourceRef("gbuffer2"), ResourceRef("depth"),
ResourceRef("shadowMap"))
rg.add_pass("PostProcess", lambda c: post_process(c)) \
.writes(ResourceRef("backbuffer")) \
.reads(ResourceRef("hdrColor"))
rg.compile()
rg.execute(ctx)
// Frame = DAG of render passes with resource dependencies
class RenderGraph {
struct Pass {
std::string name;
std::function<void(RenderContext&)> execute;
std::vector<ResourceRef> reads, writes;
};
std::vector<Pass> passes;
ResourcePool resources; // Transient (RTs, buffers)
public:
RenderGraph& addPass(std::string name, auto&& execute) {
passes.push_back({name, execute, {}, {}});
return *this;
}
RenderGraph& reads(ResourceRef r) { passes.back().reads.push_back(r); return *this; }
RenderGraph& writes(ResourceRef r) { passes.back().writes.push_back(r); return *this; }
void compile() { /* Topological sort, alias transient resources */ }
void execute(RenderContext& ctx) { for (auto& p : passes) p.execute(ctx); }
};
// Usage per frame
RenderGraph rg;
rg.addPass("ShadowMap", [&](Ctx& c) { renderShadows(c); })
.writes("shadowMap")
.reads("sceneDepth");
rg.addPass("GBuffer", [&](Ctx& c) { renderGBuffer(c); })
.writes("gbuffer0", "gbuffer1", "gbuffer2", "depth")
.reads("shadowMap");
rg.addPass("Lighting", [&](Ctx& c) { renderDeferred(c); })
.writes("hdrColor")
.reads("gbuffer0", "gbuffer1", "gbuffer2", "depth", "shadowMap");
rg.addPass("PostProcess", [&](Ctx& c) { postProcess(c); })
.writes("backbuffer")
.reads("hdrColor");
rg.compile();
rg.execute(ctx);
import java.util.*;
import java.util.function.*;
public final class RenderGraph {
public record Pass(String name, Consumer<RenderContext> execute,
List<ResourceRef> reads, List<ResourceRef> writes) {}
private final List<Pass> passes = new ArrayList<>();
private final ResourcePool resources = new ResourcePool();
public RenderGraph addPass(String name, Consumer<RenderContext> execute) {
passes.add(new Pass(name, execute, new ArrayList<>(), new ArrayList<>()));
return this;
}
public RenderGraph reads(ResourceRef r) { passes.getLast().reads.add(r); return this; }
public RenderGraph writes(ResourceRef r) { passes.getLast().writes.add(r); return this; }
public void compile() { /* Topological sort, alias transient resources */ }
public void execute(RenderContext ctx) { for (Pass p : passes) p.execute().accept(ctx); }
}
// Usage per frame
RenderGraph rg = new RenderGraph();
rg.addPass("ShadowMap", c -> renderShadows(c))
.writes(new ResourceRef("shadowMap"))
.reads(new ResourceRef("sceneDepth"));
rg.addPass("GBuffer", c -> renderGBuffer(c))
.writes(new ResourceRef("gbuffer0"), new ResourceRef("gbuffer1"),
new ResourceRef("gbuffer2"), new ResourceRef("depth"))
.reads(new ResourceRef("shadowMap"));
rg.addPass("Lighting", c -> renderDeferred(c))
.writes(new ResourceRef("hdrColor"))
.reads(new ResourceRef("gbuffer0"), new ResourceRef("gbuffer1"),
new ResourceRef("gbuffer2"), new ResourceRef("depth"),
new ResourceRef("shadowMap"));
rg.addPass("PostProcess", c -> postProcess(c))
.writes(new ResourceRef("backbuffer"))
.reads(new ResourceRef("hdrColor"));
rg.compile();
rg.execute(ctx);
using System;
using System.Collections.Generic;
public sealed class RenderGraph {
public sealed record Pass(string Name, Action<RenderContext> Execute,
List<ResourceRef> Reads, List<ResourceRef> Writes);
private readonly List<Pass> _passes = new();
private readonly ResourcePool _resources = new();
public RenderGraph AddPass(string name, Action<RenderContext> execute) {
_passes.Add(new Pass(name, execute, new(), new()));
return this;
}
public RenderGraph Reads(ResourceRef r) { _passes[^1].Reads.Add(r); return this; }
public RenderGraph Writes(ResourceRef r) { _passes[^1].Writes.Add(r); return this; }
public void Compile() { /* Topological sort, alias transient resources */ }
public void Execute(RenderContext ctx) { foreach (var p in _passes) p.Execute(ctx); }
}
// Usage per frame
var rg = new RenderGraph();
rg.AddPass("ShadowMap", c => RenderShadows(c))
.Writes(new ResourceRef("shadowMap"))
.Reads(new ResourceRef("sceneDepth"));
rg.AddPass("GBuffer", c => RenderGBuffer(c))
.Writes(new ResourceRef("gbuffer0"), new ResourceRef("gbuffer1"),
new ResourceRef("gbuffer2"), new ResourceRef("depth"))
.Reads(new ResourceRef("shadowMap"));
rg.AddPass("Lighting", c => RenderDeferred(c))
.Writes(new ResourceRef("hdrColor"))
.Reads(new ResourceRef("gbuffer0"), new ResourceRef("gbuffer1"),
new ResourceRef("gbuffer2"), new ResourceRef("depth"),
new ResourceRef("shadowMap"));
rg.AddPass("PostProcess", c => PostProcess(c))
.Writes(new ResourceRef("backbuffer"))
.Reads(new ResourceRef("hdrColor"));
rg.Compile();
rg.Execute(ctx);
ResourceRef = Struct.new(:name)
Pass = Struct.new(:name, :execute, :reads, :writes) do
def initialize(name, execute)
super(name, execute, [], [])
end
end
class RenderGraph
def initialize
@passes = []
@resources = ResourcePool.new # Transient (RTs, buffers)
end
def add_pass(name, &execute)
@passes << Pass.new(name, execute)
self
end
def reads(*refs)
@passes.last.reads.concat(refs)
self
end
def writes(*refs)
@passes.last.writes.concat(refs)
self
end
def compile
# Topological sort, alias transient resources
end
def execute(ctx)
@passes.each { |pass| pass.execute.call(ctx) }
end
end
# Usage per frame
rg = RenderGraph.new
rg.add_pass("ShadowMap") { |c| render_shadows(c) }
.writes(ResourceRef.new("shadowMap"))
.reads(ResourceRef.new("sceneDepth"))
rg.add_pass("GBuffer") { |c| render_gbuffer(c) }
.writes(ResourceRef.new("gbuffer0"), ResourceRef.new("gbuffer1"),
ResourceRef.new("gbuffer2"), ResourceRef.new("depth"))
.reads(ResourceRef.new("shadowMap"))
rg.add_pass("Lighting") { |c| render_deferred(c) }
.writes(ResourceRef.new("hdrColor"))
.reads(ResourceRef.new("gbuffer0"), ResourceRef.new("gbuffer1"),
ResourceRef.new("gbuffer2"), ResourceRef.new("depth"),
ResourceRef.new("shadowMap"))
rg.add_pass("PostProcess") { |c| post_process(c) }
.writes(ResourceRef.new("backbuffer"))
.reads(ResourceRef.new("hdrColor"))
rg.compile
rg.execute(ctx)
Shader System
Shaders complicate the picture further because the same material logic often needs many compiled variants — with and without skinning, with and without a given lighting feature, one per platform's shading language — and a shader system's job is managing that combinatorial explosion of permutations without forcing every combination to be hand-authored separately.
@dataclass
class ShaderPermutation:
base_name: str # "pbr"
defines: List[str] # "SKINNING", "NORMAL_MAP", "IBL"
def hash(self) -> int:
return hash((self.base_name, tuple(self.defines)))
class ShaderLibrary:
def __init__(self):
self._cache = {}
def get_or_compile(self, perm: ShaderPermutation) -> 'ShaderProgram':
h = perm.hash()
if h in self._cache:
return self._cache[h]
return self._compile_permutation(perm) # Async in background
def _compile_permutation(self, perm: ShaderPermutation) -> 'ShaderProgram':
pass
// Permutation system for shader variants
struct ShaderPermutation {
std::string baseName; // "pbr"
std::vector<std::string> defines; // "SKINNING", "NORMAL_MAP", "IBL"
uint64_t hash() const; // Cache key
};
class ShaderLibrary {
std::unordered_map<uint64_t, ShaderProgram> cache;
ShaderProgram* getOrCompile(const ShaderPermutation& perm) {
auto it = cache.find(perm.hash());
if (it != cache.end()) return &it->second;
return compilePermutation(perm); // Async in background
}
};
public final class ShaderPermutation {
public final String baseName; // "pbr"
public final List<String> defines; // "SKINNING", "NORMAL_MAP", "IBL"
public long hash() { return Objects.hash(baseName, defines); }
}
public final class ShaderLibrary {
private final Map<Long, ShaderProgram> cache = new ConcurrentHashMap<>();
public ShaderProgram getOrCompile(ShaderPermutation perm) {
return cache.computeIfAbsent(perm.hash(), k -> compilePermutation(perm));
}
}
using System.Collections.Generic;
public readonly record struct ShaderPermutation {
public readonly string BaseName; // "pbr"
public readonly List<string> Defines; // "SKINNING", "NORMAL_MAP", "IBL"
public ulong Hash() => HashCode.Combine(BaseName, HashCode.Combine(Defines));
}
public sealed class ShaderLibrary {
private readonly Dictionary<ulong, ShaderProgram> _cache = new();
public ShaderProgram GetOrCompile(ShaderPermutation perm) {
return _cache.GetOrAdd(perm.Hash(), _ => CompilePermutation(perm));
}
}
ShaderPermutation = Struct.new(:base_name, :defines) do
# base_name: "pbr"; defines: ["SKINNING", "NORMAL_MAP", "IBL"]
def cache_key
[base_name, *defines].hash
end
end
class ShaderLibrary
def initialize
@cache = {}
end
def get_or_compile(perm)
@cache[perm.cache_key] ||= compile_permutation(perm) # Async in background
end
private
def compile_permutation(perm); end
end
6. Physics Integration
Physics is almost always a licensed or open-source third-party library (Bullet, PhysX, Jolt) rather than something an engine team writes from scratch, which raises a specific integration problem: the physics engine has its own internal representation of the world, on its own update rate, and the game engine's ECS or scene graph has to stay synchronised with it without either side becoming tightly coupled to the other's internals.
Engine-Physics Boundary
The clean solution is a thin translation layer at the boundary: game-side transforms are pushed into the physics world before it steps, and the physics world's results are pulled back out into game-side components afterwards, with neither side ever reaching directly into the other's data structures. That boundary is also where the fixed-timestep physics tick and the variable-length render frame get reconciled, since physics generally needs deterministic, fixed-size steps regardless of how long a given frame actually took.
class PhysicsSystem:
def __init__(self):
self.world = PhysicsWorld() # PhysX, Jolt, Box2D
self.accumulator = 0.0
self.fixed_dt = 1.0 / 60.0
def step(self, dt: float) -> None:
self.accumulator += dt
while self.accumulator >= self.fixed_dt:
self.world.step(self.fixed_dt)
self.accumulator -= self.fixed_dt
self.interp_alpha = self.accumulator / self.fixed_dt
def sync_to_engine(self, ecs: 'ECS') -> None:
for entity, rigidbody, transform in ecs.view(RigidBody, Transform):
pose = rigidbody.actor.get_global_pose()
transform.pos = to_vec3(pose.p)
transform.rot = to_quat(pose.q)
def sync_from_engine(self, ecs: 'ECS') -> None:
for entity, rigidbody, transform in ecs.view(RigidBody, Transform):
if rigidbody.is_kinematic:
rigidbody.actor.set_kinematic_target(to_px_transform(transform))
class PhysicsSystem {
std::unique_ptr<PhysicsWorld> world; // PhysX, Jolt, Box2D
float accumulator = 0;
const float fixedDt = 1.0f / 60.0f;
public:
void step(float dt) {
accumulator += dt;
while (accumulator >= fixedDt) {
world->step(fixedDt);
accumulator -= fixedDt;
}
interpAlpha = accumulator / fixedDt;
}
void syncToEngine(ECS& ecs) {
for (auto [entity, rigidbody, transform] : ecs.view<RigidBody, Transform>()) {
auto pose = rigidbody.actor->getGlobalPose();
transform.pos = toVec3(pose.p);
transform.rot = toQuat(pose.q);
}
}
void syncFromEngine(ECS& ecs) {
for (auto [entity, rigidbody, transform] : ecs.view<RigidBody, Transform>()) {
if (rigidbody.isKinematic) {
rigidbody.actor->setKinematicTarget(toPxTransform(transform));
}
}
}
};
public final class PhysicsSystem {
private final PhysicsWorld world;
private float accumulator = 0;
private final float fixedDt = 1f / 60f;
public void step(float dt) {
accumulator += dt;
while (accumulator >= fixedDt) {
world.step(fixedDt);
accumulator -= fixedDt;
}
interpAlpha = accumulator / fixedDt;
}
public void syncToEngine(ECS ecs) {
for (var e : ecs.view(RigidBody.class, Transform.class)) {
var pose = e.get(RigidBody.class).actor.getGlobalPose();
e.get(Transform.class).pos = toVec3(pose.p);
e.get(Transform.class).rot = toQuat(pose.q);
}
}
public void syncFromEngine(ECS ecs) {
for (var e : ecs.view(RigidBody.class, Transform.class)) {
if (e.get(RigidBody.class).isKinematic) {
e.get(RigidBody.class).actor.setKinematicTarget(toPxTransform(e.get(Transform.class)));
}
}
}
}
public sealed class PhysicsSystem {
private readonly PhysicsWorld _world;
private float _accumulator = 0;
private const float FixedDt = 1f / 60f;
public void Step(float dt) {
_accumulator += dt;
while (_accumulator >= FixedDt) {
_world.Step(FixedDt);
_accumulator -= FixedDt;
}
InterpAlpha = _accumulator / FixedDt;
}
public void SyncToEngine(ECS ecs) {
foreach (var (entity, rigidbody, transform) in ecs.View<ibrationBody, Transform>()) {
var pose = rigidbody.Actor.GetGlobalPose();
transform.Pos = ToVec3(pose.P);
transform.Rot = ToQuat(pose.Q);
}
}
public void SyncFromEngine(ECS ecs) {
foreach (var (entity, rigidbody, transform) in ecs.View<RigidBody, Transform>()) {
if (rigidbody.IsKinematic) {
rigidbody.Actor.SetKinematicTarget(ToPxTransform(transform));
}
}
}
}
class PhysicsSystem
FIXED_DT = 1.0 / 60.0
attr_reader :interp_alpha
def initialize
@world = PhysicsWorld.new # PhysX, Jolt, Box2D
@accumulator = 0.0
end
def step(dt)
@accumulator += dt
while @accumulator >= FIXED_DT
@world.step(FIXED_DT)
@accumulator -= FIXED_DT
end
@interp_alpha = @accumulator / FIXED_DT
end
def sync_to_engine(ecs)
ecs.view(RigidBody, Transform) do |_entity, rigidbody, transform|
pose = rigidbody.actor.global_pose
transform.pos = to_vec3(pose.p)
transform.rot = to_quat(pose.q)
end
end
def sync_from_engine(ecs)
ecs.view(RigidBody, Transform) do |_entity, rigidbody, transform|
if rigidbody.kinematic?
rigidbody.actor.kinematic_target = to_px_transform(transform)
end
end
end
end
Collision Callbacks
The other half of the boundary runs in the opposite direction: gameplay code needs to react when two physics bodies touch, without needing to know anything about the physics engine's internal broad-phase or narrow-phase collision detection. A callback layer translates the physics engine's raw collision events into game-level events — "this entity took damage," "this trigger volume was entered" — keeping gameplay code decoupled from whichever physics library happens to be underneath it.
@dataclass
class CollisionEvent:
a: Entity
b: Entity
point: Vec3
normal: Vec3
impulse: float
class PhysicsSystem:
def __init__(self):
self.collisions = EventBus[CollisionEvent]()
def on_contact(self, pair: ContactPair) -> None:
a = pair.shapes[0].get_user_data(Entity)
b = pair.shapes[1].get_user_data(Entity)
for contact in pair.contacts:
self.collisions.publish(CollisionEvent(a, b, contact.point, contact.normal, contact.impulse))
# Gameplay subscribes
collisions.subscribe(lambda e: (
e.a.get(Health).damage(10) if e.a.has(Player) and e.b.has(Enemy) else None,
e.b.get(Audio).play("hit") if e.a.has(Player) and e.b.has(Enemy) else None
))
struct CollisionEvent {
Entity a, b;
Vec3 point, normal;
float impulse;
};
class PhysicsSystem {
EventBus<CollisionEvent> collisions;
void onContact(const ContactPair& pair) {
Entity a = pair.shapes[0].getUserData<Entity>();
Entity b = pair.shapes[1].getUserData<Entity>();
for (auto& contact : pair.contacts) {
collisions.publish({a, b, contact.point, contact.normal, contact.impulse});
}
}
};
// Gameplay subscribes
collisions.subscribe([](const CollisionEvent& e) {
if (e.a.has<Player>() && e.b.has<Enemy>()) {
e.a.get<Health>().damage(10);
e.b.get<Audio>().play("hit");
}
});
public record CollisionEvent(Entity a, Entity b, Vec3 point, Vec3 normal, float impulse) {}
public final class PhysicsSystem {
private final EventBus<CollisionEvent> collisions = new EventBus<>();
public void onContact(ContactPair pair) {
Entity a = pair.shapes[0].getUserData(Entity.class);
Entity b = pair.shapes[1].getUserData(Entity.class);
for (var contact : pair.contacts) {
collisions.publish(new CollisionEvent(a, b, contact.point, contact.normal, contact.impulse));
}
}
}
// Gameplay subscribes
collisions.subscribe(e -> {
if (e.a().has(Player.class) && e.b().has(Enemy.class)) {
e.a().get(Health.class).damage(10);
e.b().get(Audio.class).play("hit");
}
});
public readonly record struct CollisionEvent(
Entity A, Entity B, Vec3 Point, Vec3 Normal, float Impulse
);
public sealed class PhysicsSystem {
private readonly EventBus<CollisionEvent> _collisions = new();
public void OnContact(ContactPair pair) {
Entity a = pair.Shapes[0].GetUserData<Entity>();
Entity b = pair.Shapes[1].GetUserData<Entity>();
foreach (var contact in pair.Contacts) {
_collisions.Publish(new CollisionEvent(a, b, contact.Point, contact.Normal, contact.Impulse));
}
}
}
// Gameplay subscribes
_collisions.Subscribe(e => {
if (e.A.Has<Player>() && e.B.Has<Enemy>()) {
e.A.Get<Health>().Damage(10);
e.B.Get<Audio>().Play("hit");
}
});
CollisionEvent = Struct.new(:a, :b, :point, :normal, :impulse)
class PhysicsSystem
attr_reader :collisions
def initialize
@collisions = EventBus.new
end
def on_contact(pair)
a = pair.shapes[0].user_data
b = pair.shapes[1].user_data
pair.contacts.each do |contact|
@collisions.publish(
CollisionEvent.new(a, b, contact.point, contact.normal, contact.impulse)
)
end
end
end
# Gameplay subscribes
collisions.subscribe do |e|
if e.a.has?(Player) && e.b.has?(Enemy)
e.a.get(Health).damage(10)
e.b.get(Audio).play("hit")
end
end
7. SGA Globe Rendering — Cross-Reference
Side Note: Our SGA Globe Rendering project (SGA Globe Rendering with Interval Arithmetic) demonstrates a specialised engine architecture for planetary-scale rendering:
Key Architectural Decisions from SGA Globe
Every general-purpose renderer described above is built around approximating a scene closely enough, fast enough — heightmaps, ray-marched terrain, shadow maps. SGA Globe is worth studying precisely because it takes the opposite bet for one specific problem (planetary-scale terrain): rather than approximating, it represents the world implicitly and solves for exact intersections using interval arithmetic, trading a more specialised mathematical toolkit for correctness a heightmap-based approach can't offer at planetary scale.
| Traditional Engine | SGA Globe Approach |
|---|---|
| Ray-marching for terrain | Interval arithmetic bisection — mathematically rigorous root finding |
| CPU-side LOD | GPU-driven LOD via interval subdivision |
| Double precision | Spherical Geometric Algebra (SGA) — coordinate-free maths |
| Heightmap sampling | Implicit SDF — length(p) - (R + h(θ,φ)) |
| Shadow maps | Analytic visibility via interval bounds |
Applicable Lessons for General Engines
None of these ideas are useful only for planetary terrain, though — each one generalises to a specific, recognisable problem a general-purpose engine runs into elsewhere, which is why the specialised case is worth a general engine team's attention at all.
- Implicit representations win for infinite/procedural content
- Interval arithmetic provides error bounds → automatic LOD
- Coordinate-free maths (GA) eliminates singularities at poles
- GPU-driven pipelines shift work from CPU to compute shaders
- Mathematical rigour enables new rendering techniques (exact intersections)
# SGA Globe shader — interval bisection (from paper)
class Interval:
def __init__(self, lo: float, hi: float):
self.lo, self.hi = lo, hi
def mid(self) -> 'Interval':
return Interval((self.lo + self.hi) * 0.5, (self.lo + self.hi) * 0.5)
def trace_globe(ro: Vec3, rd: Vec3) -> float:
t = Interval(near, far)
for _ in range(32): # Sign change detection
t_mid = t.mid()
p = ro + rd * t_mid.lo
d = sdf(p) # Implicit SDF with interval arithmetic
if d.lo <= 0.0 and d.hi >= 0.0:
t = Interval(t.lo, t_mid.hi) # Root in lower half
else:
t = Interval(t_mid.lo, t.hi)
# Bisection refinement
for _ in range(20):
t_mid = (t.lo + t.hi) * 0.5
d = sdf_scalar(ro + rd * t_mid)
if d <= 0.0: t.hi = t_mid
else: t.lo = t_mid
return t.hi
// SGA Globe shader — interval bisection (from paper)
struct Interval { float lo, hi; };
inline Interval mid(const Interval& t) { return {(t.lo + t.hi) * 0.5f, (t.lo + t.hi) * 0.5f}; }
float traceGlobe(Vec3 ro, Vec3 rd) {
Interval t = {near, far};
for (int i = 0; i < 32; ++i) { // Sign change detection
Interval t_mid = mid(t);
Vec3 p = ro + rd * t_mid.lo;
Interval d = sdf(p); // Implicit SDF with interval arithmetic
if (d.lo <= 0.0f && d.hi >= 0.0f) t = {t.lo, t_mid.hi};
else t = {t_mid.lo, t.hi};
}
// Bisection refinement
for (int i = 0; i < 20; ++i) {
float t_mid = (t.lo + t.hi) * 0.5f;
float d = sdf_scalar(ro + rd * t_mid);
if (d <= 0.0f) t.hi = t_mid; else t.lo = t_mid;
}
return t.hi;
}
public final class SGAInterval {
public final float lo, hi;
public SGAInterval(float lo, float hi) { this.lo = lo; this.hi = hi; }
public SGAInterval mid() { return new SGAInterval((lo + hi) * 0.5f, (lo + hi) * 0.5f); }
}
public static float traceGlobe(Vec3 ro, Vec3 rd) {
SGAInterval t = new SGAInterval(near, far);
for (int i = 0; i < 32; ++i) { // Sign change detection
var tMid = t.mid();
Vec3 p = ro.add(rd.mul(tMid.lo));
var d = sdf(p); // Implicit SDF with interval arithmetic
if (d.lo <= 0 && d.hi >= 0) t = new SGAInterval(t.lo, tMid.hi);
else t = new SGAInterval(tMid.lo, t.hi);
}
// Bisection refinement
for (int i = 0; i < 20; ++i) {
float tMid = (t.lo + t.hi) * 0.5f;
float d = sdfScalar(ro.add(rd.mul(tMid)));
if (d <= 0) t.hi = tMid; else t.lo = tMid;
}
return t.hi;
}
public readonly record struct SGAInterval(float Lo, float Hi) {
public SGAInterval Mid() => new((Lo + Hi) * 0.5f, (Lo + Hi) * 0.5f);
}
public static float TraceGlobe(Vec3 ro, Vec3 rd) {
var t = new SGAInterval(near, far);
for (int i = 0; i < 32; ++i) { // Sign change detection
var tMid = t.Mid();
var p = ro + rd * tMid.Lo;
var d = SDF(p); // Implicit SDF with interval arithmetic
if (d.Lo <= 0 && d.Hi >= 0) t = new SGAInterval(t.Lo, tMid.Hi);
else t = new SGAInterval(tMid.Lo, t.Hi);
}
// Bisection refinement
for (int i = 0; i < 20; ++i) {
float tMid = (t.Lo + t.Hi) * 0.5f;
float d = SdfScalar(ro + rd * tMid);
if (d <= 0) t = new SGAInterval(t.Lo, tMid);
else t = new SGAInterval(tMid, t.Hi);
}
return t.Hi;
}
# SGA Globe shader — interval bisection (from paper)
Interval = Struct.new(:lo, :hi) do
def mid
m = (lo + hi) * 0.5
Interval.new(m, m)
end
end
def trace_globe(ro, rd, near, far)
t = Interval.new(near, far)
32.times do # Sign change detection
t_mid = t.mid
p = ro + rd * t_mid.lo
d = sdf(p) # Implicit SDF with interval arithmetic
t = if d.lo <= 0.0 && d.hi >= 0.0
Interval.new(t.lo, t_mid.hi) # Root in lower half
else
Interval.new(t_mid.lo, t.hi)
end
end
# Bisection refinement
20.times do
t_mid = (t.lo + t.hi) * 0.5
d = sdf_scalar(ro + rd * t_mid)
d <= 0.0 ? t.hi = t_mid : t.lo = t_mid
end
t.hi
end
8. Scripting Integration
Compiling and re-linking a game just to tweak an enemy's attack timing or a UI layout is a slow feedback loop that designers and scripters shouldn't have to live inside. Embedding a scripting language addresses this directly: gameplay logic that changes often lives in script, is reloaded without a full engine rebuild, and calls into performance-critical engine systems (rendering, physics, audio) written in the compiled language underneath.
Lua Integration (sol2)
Lua is the most common choice for this role because it's small, fast to embed, and easy to sandbox — a script shouldn't be able to reach outside the specific engine API it's given access to. A binding library like sol2 handles the tedious, error-prone part of exposing C++ types and functions to Lua directly, so engine code can register a class once and have it usable from script with (mostly) natural syntax on both sides.
# Lua via sol2 (C++ only - shown for reference)
# Python embedding alternative:
import lua54
from functools import partial
class ScriptSystem:
def __init__(self):
self.lua = lua54.new_state()
self.lua.open_libs(lua54.base, lua54.math, lua54.table)
self.entity_scripts = {}
# Bind engine API
self._bind_entity_api()
self._bind_factory_api()
self._bind_ecs_api()
def _bind_entity_api(self) -> None:
@lua54.function(self.lua)
def entity_get(entity, comp_type):
return ecs.get_component(entity, comp_type)
@lua54.function(self.lua)
def entity_add(entity, comp_type, *args):
return ecs.add_component(entity, comp_type, *args)
def _bind_factory_api(self) -> None:
@lua54.function(self.lua)
def spawn(prefab: str, pos: tuple) -> int:
return factory.create_from_prefab(prefab, pos)
def run_script(self, entity: int, script_path: str) -> None:
script = load_script(script_path)
env = self.lua.new_table()
env["self"] = entity
script(env)
#include <sol/sol.hpp>
class ScriptSystem {
sol::state lua;
std::unordered_map<Entity, sol::table> entityScripts;
public:
ScriptSystem() {
lua.open_libraries(sol::lib::base, sol::lib::math, sol::lib::table);
// Bind engine API
lua.new_usertype<Entity>("Entity",
"get", &Entity::getComponent,
"add", &Entity::addComponent,
"destroy", &Entity::destroy
);
lua.set_function("spawn", [this](const std::string& prefab, Vec3 pos) {
return factory.createFromPrefab(prefab, pos);
});
lua.set_function("find", [this](const std::string& tag) {
return ecs.findByTag(tag);
});
}
void runScript(Entity e, const std::string& scriptPath) {
auto script = loadScript(scriptPath);
sol::environment env(lua, sol::create, lua.globals());
env["self"] = e;
script(env);
}
};
// LuaJ integration
import org.luaj.*;
import org.luaj.lib.*;
public class ScriptSystem {
private final Globals globals = JsePlatform.standardGlobals();
private final Map<Entity, LuaTable> entityScripts = new HashMap<>();
public ScriptSystem() {
globals.load(new BaseLib());
globals.load(new MathLib());
globals.load(new TableLib());
// Bind engine API
globals.set("spawn", new TwoArgFunction() {
public LuaValue call(LuaValue prefab, LuaValue pos) {
return LuaValue.valueOf(factory.createFromPrefab(prefab.checkString(), toVec3(pos)));
}
});
globals.set("find", new OneArgFunction() {
public LuaValue call(LuaValue tag) {
return LuaValue.valueOf(ecs.findByTag(tag.checkString()));
}
});
}
public void runScript(Entity e, String scriptPath) {
LuaValue script = globals.loadfile(scriptPath);
var env = new LuaTable(globals);
env.set("self", LuaValue.valueOf(e.getId()));
script.call(env);
}
}
// MoonSharp integration
using MoonSharp.Interpreter;
public class ScriptSystem {
private readonly Script _lua = new Script(CoreModules.Preset_HardSandbox);
private readonly Dictionary<Entity, Table> _entityScripts = new();
public ScriptSystem() {
_lua.Globals["spawn"] = (Func<string, Vec3, int>)((prefab, pos) =>
Factory.CreateFromPrefab(prefab, pos));
_lua.Globals["find"] = (Func<string, int>)(tag =>
Ecs.FindByTag(tag));
}
public void RunScript(Entity e, string scriptPath) {
string script = File.ReadAllText(scriptPath);
Table env = new Table(_lua);
env["self"] = e.Id;
_lua.DoString(script, env);
}
}
# Lua embedding via the rufus-lua gem (wraps liblua)
require "rufus-lua"
class ScriptSystem
def initialize
@lua = Rufus::Lua::State.new
@entity_scripts = {}
# Bind engine API
bind_entity_api
bind_factory_api
end
def run_script(entity, script_path)
@lua["self"] = entity
@lua.eval(File.read(script_path))
end
private
def bind_entity_api
@lua.function("entity_get") do |entity, comp_type|
ecs.get_component(entity, comp_type)
end
@lua.function("entity_add") do |entity, comp_type, *args|
ecs.add_component(entity, comp_type, *args)
end
end
def bind_factory_api
@lua.function("spawn") do |prefab, pos|
factory.create_from_prefab(prefab, pos)
end
end
end
9. Editor Architecture
An engine's editor is not, in a well-built engine, a separate program that happens to produce files the runtime reads — it's the same runtime, running the same systems, with an extra layer of tooling UI drawn on top and a few extra affordances (undo, gizmos, live property editing) enabled. Building it any other way risks the editor and the shipped game silently drifting apart in behaviour, which is a much worse bug class than a slow editor.
ImGui-Based Editor
Dear ImGui is the near-universal choice for this kind of tooling UI because it's an immediate-mode library: panels are just drawn fresh every frame from whatever the current engine state is, with no separate retained UI tree to keep in sync by hand. That property maps naturally onto an editor's actual job — reflecting live engine state back at the developer — in a way a traditional retained-mode UI toolkit has to work much harder to achieve.
import imgui
class Editor:
def __init__(self, ecs):
self.ecs = ecs
self.show_inspector = True
self.show_asset_browser = True
self.show_profiler = False
self.selected_entity = None
def draw(self) -> None:
self._draw_menu_bar()
if self.show_inspector: self._draw_entity_inspector()
if self.show_asset_browser: self._draw_asset_browser()
if self.show_profiler: self._draw_profiler()
self._draw_viewport()
def _draw_entity_inspector(self) -> None:
imgui.begin("Inspector", lambda: setattr(self, 'show_inspector', False))
if self.selected_entity:
imgui.text(f"Entity: {self.selected_entity}")
for comp_type, comp in self.ecs.get_components(self.selected_entity).items():
if imgui.collapsing_header(comp_type.__name__):
self._reflect_draw(comp)
if imgui.button("Add Component"):
imgui.open_popup("AddComponent")
self._draw_add_component_popup()
imgui.end()
def _draw_viewport(self) -> None:
imgui.begin("Viewport")
# Render to offscreen texture, display as ImGui::Image
# Gizmo interaction (ImGuizmo)
if self.selected_entity:
self._draw_gizmo(self.selected_entity)
imgui.end()
class Editor {
bool showEntityInspector = true;
bool showAssetBrowser = true;
bool showProfiler = false;
Entity selectedEntity = INVALID;
public:
void draw() {
drawMenuBar();
if (showEntityInspector) drawEntityInspector();
if (showAssetBrowser) drawAssetBrowser();
if (showProfiler) drawProfiler();
drawViewport();
}
void drawEntityInspector() {
ImGui::Begin("Inspector", &showEntityInspector);
if (selectedEntity != INVALID) {
ImGui::Text("Entity: %llu", selectedEntity);
// Auto-generate from component reflection
for (auto& [type, comp] : ecs.getComponents(selectedEntity)) {
if (ImGui::CollapsingHeader(type.name())) {
reflectDraw(comp); // Auto-UI from reflection
}
}
if (ImGui::Button("Add Component")) {
ImGui::OpenPopup("AddComponent");
}
drawAddComponentPopup();
}
ImGui::End();
}
void drawViewport() {
ImGui::Begin("Viewport");
ImVec2 size = ImGui::GetContentRegionAvail();
ImGui::Image(renderTargetSRV, ImVec2(size.x, size.y));
// Gizmo interaction (ImGuizmo)
if (selectedEntity != INVALID) {
drawGizmo(selectedEntity);
}
ImGui::End();
}
};
// ImGui Java bindings
import org.imgui.*;
public class Editor {
private boolean showInspect = true;
private boolean showAssets = true;
private boolean showProfiler = false;
private Entity selected = null;
public void draw() {
drawMenuBar();
if (showInspect) drawEntityInspector();
if (showAssets) drawAssetBrowser();
if (showProfiler) drawProfiler();
drawViewport();
}
private void drawEntityInspector() {
ImGui.begin("Inspector", () -> showInspect = false);
if (selected != null) {
ImGui.text("Entity: " + selected.id());
for (var entry : ecs.getComponents(selected).entrySet()) {
if (ImGui.collapsingHeader(entry.getKey().getSimpleName())) {
reflectDraw(entry.getValue());
}
}
if (ImGui.button("Add Component"))
ImGui.openPopup("AddComponent");
drawAddComponentPopup();
}
ImGui.end();
}
private void drawViewport() {
ImGui.begin("Viewport");
ImVec2 size = ImGui.getContentRegionAvail();
ImGui.image(renderTargetSRV, new ImVec2(size.x, size.y));
if (selected != null) drawGizmo(selected);
ImGui.end();
}
}
using ImGuiNET;
public class Editor {
bool _showInspector = true;
bool _showAssets = true;
bool _showProfiler = false;
Entity? _selectedEntity = null;
public void Draw() {
DrawMenuBar();
if (_showInspector) DrawEntityInspector();
if (_showAssets) DrawAssetBrowser();
if (_showProfiler) DrawProfiler();
DrawViewport();
}
void DrawEntityInspector() {
ImGui.Begin("Inspector", ref _showInspector);
if (_selectedEntity != null) {
ImGui.Text($"Entity: {_selectedEntity.Id}");
foreach (var (type, comp) in _ecs.GetComponents(_selectedEntity)) {
if (ImGui.CollapsingHeader(type.Name)) {
ReflectDraw(comp);
}
}
if (ImGui.Button("Add Component"))
ImGui.OpenPopup("AddComponent");
DrawAddComponentPopup();
}
ImGui.End();
}
void DrawViewport() {
ImGui.Begin("Viewport");
var size = ImGui.GetContentRegionAvail();
ImGui.Image(renderTargetSRV, new System.Numerics.Vector2(size.X, size.Y));
if (_selectedEntity != null) DrawGizmo(_selectedEntity);
ImGui.End();
}
}
# Dear ImGui via the ruby-imgui gem
require "imgui"
class Editor
def initialize(ecs)
@ecs = ecs
@show_inspector = true
@show_asset_browser = true
@show_profiler = false
@selected_entity = nil
end
def draw
draw_menu_bar
draw_entity_inspector if @show_inspector
draw_asset_browser if @show_asset_browser
draw_profiler if @show_profiler
draw_viewport
end
private
def draw_entity_inspector
ImGui::Begin("Inspector")
if @selected_entity
ImGui::Text("Entity: #{@selected_entity}")
@ecs.components(@selected_entity).each do |comp_type, comp|
reflect_draw(comp) if ImGui::CollapsingHeader(comp_type.name)
end
ImGui::OpenPopup("AddComponent") if ImGui::Button("Add Component")
draw_add_component_popup
end
ImGui::End()
end
def draw_viewport
ImGui::Begin("Viewport")
# Render to offscreen texture, display as ImGui::Image
# Gizmo interaction (ImGuizmo)
draw_gizmo(@selected_entity) if @selected_entity
ImGui::End()
end
end
10. Build Pipeline
Shipping a game across five-plus platforms from one codebase means the build pipeline has to describe, in one place, everything that differs between them: which defines and sanitizers apply to a debug build versus a shipping build, what texture and audio formats each platform actually wants, and what final package format each storefront expects. Configuration-as-data, as in the example below, keeps all of that declarative and versioned alongside the code, rather than scattered across ad-hoc build scripts that drift from what the platforms actually require.
# build.yaml
platforms: [windows, linux, macos, android, ios]
configurations:
debug:
defines: [DEBUG, TRACY_ENABLE]
sanitize: [address, undefined]
release:
defines: [NDEBUG]
optimize: speed
lto: full
shipping:
defines: [NDEBUG, SHIPPING]
optimize: speed
lto: full
strip: true
asset_pipeline:
textures:
- format: BC7 (desktop), ASTC (mobile)
mipmaps: true
compress: true
meshes:
- optimize: meshopt
quantize: [position: 16, normal: 8, uv: 16]
generate_tangents: true
audio:
- format: Opus (music), ADPCM (sfx)
sample_rate: 48000
packaging:
windows: [exe, dlls, assets.pak]
linux: [elf, .so, assets.pak]
macos: [.app bundle]
android: [.aab, .apk]
ios: [.ipa]
Resources
Everything on this page is a survey of well-established techniques rather than an exhaustive treatment of any one of them; each subsection below points at where to actually go deeper, whether that's a standard textbook, a real shipped codebase to read, or the original conference talk a technique came from.
Books
- Game Engine Architecture — Jason Gregory (3rd ed.), the general survey this page's own structure loosely follows — see the References section for the full citation.
- Real-Time Rendering — Akenine-Möller, Haines, & Hoffman, the standard reference for the rendering techniques in Section 5.
- Real-Time Collision Detection — Christer Ericson, covering broad-phase/narrow-phase collision and continuous collision detection referenced in Section 6.
- Game Programming Patterns — Robert Nystrom, the companion piece to the patterns covered on the Game Programming Patterns page.
- Foundations of Game Engine Development, Vol. 1: Mathematics — Eric Lengyel, covering the 3D maths underlying the renderer and physics sections.
Open Source Engines (Study)
Reading a real engine's source is a faster way to see these architectural patterns in practice than any single article, because a shipped engine has already had to make every one of the trade-offs described above concretely, in code.
| Engine | Language | Notable For |
|---|---|---|
| Godot | C++/GDScript | Node-based, open source |
| Bevy | Rust | ECS, data-driven |
| Unreal | C++ | Industry standard, source available |
| O3DE | C++ | Modular, AWS-backed |
| bgfx | C++ | Cross-platform renderer |
| EnTT | C++ | Fast ECS |
| flecs | C | ECS with queries |
Papers & Talks
Several of the specific designs described above trace back to a single named conference talk rather than a general body of practice — these are worth watching directly, since a slide deck built around one shipped engine's real numbers is often more concrete than any textbook chapter.
- O'Donnell, Y. (2017). "FrameGraph: Extensible Rendering Architecture in Frostbite." GDC 2017 — the talk the render graph in Section 5 is drawn from.
- Aaltonen, S. & Haar, U. (2015). "GPU-Driven Rendering Pipelines." SIGGRAPH 2015 course, Advances in Real-Time Rendering in Games.
- Gyrling, C. (2015). "Parallelizing the Naughty Dog Engine Using Fibers." GDC 2015 — a real shipped fiber-based job system, referenced in Section 3.
Summary: Engine Layers
Stripped of the detail in each of the ten sections above, the whole engine reduces to the same four-layer stack introduced at the top of the page. Every concrete system covered — job system, ECS, renderer, physics, scripting, editor — is a runtime or infrastructure system sitting on top of the platform abstraction layer, and none of them should ever need to reach below the layer directly beneath them.
┌─────────────────────────────────────────────┐
│ GAME LOGIC │ (Your game code)
├─────────────────────────────────────────────┤
│ ENGINE RUNTIME │
│ ECS │ Physics │ Renderer │ Audio │ Script │ (Systems)
├─────────────────────────────────────────────┤
│ CORE INFRASTRUCTURE │
│ Job System │ Memory │ Assets │ Config │ (Foundation)
├─────────────────────────────────────────────┤
│ PLATFORM ABSTRACTION LAYER │
│ Window │ Input │ GPU │ Audio │ FS │ Net │ (Platform)
└─────────────────────────────────────────────┘
Rule of thumb: The engine should make the common case fast and the complex case possible. Don't over-engineer — build what your game needs, refactor when patterns emerge.
References
- Gregory, J. (2018). Game Engine Architecture (3rd ed.). CRC Press. https://www.gameenginebook.com/