Presence, discovery, and non-blocking spawn: composing PatLang processes at runtime

A five-part answer to a question posed mid-session: could a running PatLang program discover other running PatLang programs it didn't already know about, learn what they offer, and treat them as callable objects — "compose independently-running processes into a larger system at runtime, discovered rather than pre-wired"? Everything below is real, working code, verified against real running processes at every stage, not a design sketch. Four small library modules (self_hosting/lib/task_registry.patlang, signal_discovery.patlang, plus the existing signals.patlang and queue.patlang they build on), one new engine primitive (non-blocking process spawn), and a live browser dashboard.

The foundation already there: signals and the message queue

Two pieces of existing infrastructure turned out to be exactly the right building blocks, neither built for this purpose originally. Runtime signals (self_hosting/lib/signals.patlang) already let one running PatLang process send a named message to another over localhost TCP, with request/response semantics (signal_query) built on the same when/emit event mechanism every other part of the language already uses. And a durable, file-backed message queue (self_hosting/lib/queue.patlang) already existed for a completely different purpose (a saga-pattern work queue with resumable, crash-safe intent/attempt/outcome tracking).

The gap was discovery: signal_query(port, ...) needs an already-known port. Nothing let a process find out what to ask in the first place. Two designs were considered for that specifically. The first — a well-known TCP port acting as a directory, tasks announcing themselves to it — had a real problem: PatLang has no try/catch, so a plain connection attempt to a port nobody's listening on is fatal, meaning any kind of port-scanning discovery would abort the whole program on the first empty port it tried. The message queue sidestepped that entirely — reading a file that isn't there just returns nothing, no fatal error possible. That reframing came from a direct question rather than being the first idea reached for: "I guess we could use the message queue, actually...?"

Part 1: a presence beacon with a TTL, built from primitives that already existed

A task that wants to be discoverable periodically writes "I am here" to a well-known message-queue topic, overwriting its own previous entry each time rather than accumulating a log of them — a presence beacon needs only its latest state. A discoverer reads every currently-fresh beacon and ignores anything older than a given staleness threshold:

make a function called tr_announce takes task_name, description returns done
  let payload = ("" + now_ms()) + q_tab() + description
  q_write(tr_topic_for(task_name), [[0, "pending", payload]])
  return true
end

make a function called tr_discover_all takes max_age_ms returns tasks
  ... list_dir(q_dir()), filter to this registry's files, read each,
  ... exclude anything whose timestamp is older than max_age_ms
end

No new host function was needed at all — now_ms(), list_dir(), and the queue library's own queue_writer_topic (built for a completely different multi-writer-race-avoidance reason, reused here unchanged) were already enough. A task that crashes simply stops refreshing its beacon and quietly ages out of discovery on its own — no explicit deregistration step, no separate monitor process watching for dead tasks, no new failure mode to design around. The question that led here, verbatim: "Wondering if live programs could have a periodic update to the message queue, retracting previous 'I am here' registrations and providing a new one, and consumers only use messages on the queue that are recent - does the MQ have an optional TTL at the moment?" — checked directly, the queue library had no such concept; this is the layer that adds it, on top, without changing the queue library itself.

Part 2: capability description folded into the same beacon

Rather than a separate query round-trip ("are you there?" then "what can you do?"), a task's beacon announcement carries its capability list directly — discovering that a task exists and learning what it offers become the same single read:

signal_announce("weather_bot", 9500, [
  ["forecast", "get tomorrow's forecast for a city"],
  ["alert", "subscribe to severe weather alerts"]
])

let found = signal_discover_one("weather_bot", 60000)
# found = [port, actions] -- actions = [[name, description], ...]

The encoding uses two multi-character markers, not single characters, to separate actions from each other and a name from its description — a lesson learned the hard way earlier in the same broader session while building compiler diagnostics, where a single-character delimiter collided with real content the moment a message happened to echo that exact character back. Applying that lesson here before it could bite a second time, rather than rediscovering it.

The honest limitation, stated plainly rather than glossed over: a capability description is entirely self-reported. Nothing verifies it's accurate. Raised directly, unprompted, as the design was being scoped: "worth being upfront that you will want to use with care as a bad actor could write programs which falsely advertise what they do." A discovered task's advertised abilities are a claim, not a guarantee — a caller that genuinely needs assurance has to verify by trying the capability, a separate, harder problem this doesn't attempt to solve.

Part 3: wrapping a discovered task as a callable object

PatLang's new/send/get turned out not to be real user-extensible method dispatch — send's only built-in verb is "set", plus a handful of Ruby-style primitive methods. There's no way to hang a custom handler off send itself. So "wrap it as an object you can call" means: store the port and action list as ordinary properties on a named object, and provide a plain function — not routed through send — that reads those properties back out and does the actual dispatch:

signal_wrap("george", 9600, actions)
let reply = signal_call("george", "greet", "Pat")
# -> a real signal_query round trip to whatever port "george" was wrapped with

signal_call checks the requested action against the object's own stored list before ever attempting the network call — an unknown action returns a clear "ERROR: ..." string naming the object and listing what it actually advertised, rather than either a fatal connection attempt or a silent no-op. This whole slice was verified against two genuinely separate, real running processes, not simulated: one holding a port and answering a real "greet" signal, a second discovering it, wrapping it, and getting a real reply back — REPLY: hello, Pat, actually printed by actually running the two programs concurrently.

Part 4: crude fit-matching, with its limits demonstrated rather than hidden

Given a need, which discovered task (if any) can help? Deliberately narrow: case-sensitive substring matching against an action's name and description, nothing smarter — explicitly not semantic reasoning, not fuzzy matching, not a model call, matching the same algorithmic-only discipline used everywhere else in this codebase (the induction engine, the compiler-diagnostics suggestion table). The test suite proves the boundary directly rather than just asserting the happy path: a weather bot offering "subscribe to severe weather alerts" is correctly found by the keyword "alert" or "severe", but a search for "meteorology" — a term a human would obviously connect to a weather bot, but one that never appears verbatim — correctly finds nothing. That's not a bug; it's the honest, stated shape of what "crude" means here, demonstrated in the test suite rather than just claimed in a comment.

Part 5: a live dashboard, for free

Once discovery returns structured data, a "what's alive right now" dashboard is just that data rendered as a page — reusing an existing native-PatLang HTTP server pattern unchanged (GET / for the page, GET /data for live JSON, a small JS refresh loop). Verified live, twice over — once interpreted, once compiled — against an actually-running task, showing its real port and action list, refreshed while a separate process was simultaneously discovering and calling that same task through the normal path. The idea arrived as a natural next thought once discovery worked: "we could build a patlang task monitor dashboard which shows what is alive."

The missing piece: nothing could actually start another process

Every verification up to this point needed two processes running concurrently — and PatLang itself had no way to make that happen. exec_capture blocks until the child exits; nothing else could launch a process and keep going. Every live test in Parts 1 through 5 had to be manually orchestrated from outside PatLang entirely, two separate scripts started as background jobs. That gap, hit twice in a row, prompted the obvious question: "We should consider the pros and cons of implementing a non-blocking process-spawn primitive."

The real tension, thought through before writing any code: PatLang has no try/catch and no destructors. A background process a program forgets to explicitly wait on or kill leaks with no possible automatic cleanup — not "harder to get right" the way an equivalent gap in Rust would be (Rust at least has Drop to hook cleanup into), genuinely unrecoverable at the language level. That risk doesn't go away by adding the primitive carefully; it's inherent to the primitive existing at all. Scoped and built anyway, with that risk documented plainly rather than pretended away:

let pid = spawn("some_program.exe", "arg1", "arg2")
if is_alive(pid) then ... end
let exit_code = wait(pid)     # blocks until it exits
let ok = kill(pid)            # best-effort; killing an already-dead process still succeeds

And the readiness-confirmation idea that prompted scoping this in the first place — "I'd like spawned processes to tell the parent what their IP and port are as confirmation they are up" — turned out to need no new mechanism whatsoever. The presence beacon from Part 1 already answers exactly that question. A parent spawns a child, then polls both is_alive (to catch an early crash before the child ever gets a chance to announce itself) and the beacon-based discovery (to catch a successful start) in one loop:

make a function called wait_for_ready takes task_name, pid, timeout_ms returns result
  # polls is_alive(pid) AND signal_discover_one(task_name, ...) together --
  # neither alone can distinguish "still starting up" from "already died"
  ...
end

Proven end to end, and for the first time in this whole arc, genuinely self-contained — no external orchestration needed at all: a parent PatLang program spawns a child, waits for it to confirm readiness via its own beacon announcement, wraps it, calls it, and gets a real reply back, all from one script.

One real bug came directly out of watching that demo run: a stray "true" appeared, interleaved oddly between two of the parent's own print statements. Traced to a detail of how the primitive was first built — a spawned child, by the underlying platform default, shares its parent's own output stream rather than getting its own. The child's own interpreter was auto-printing its own final result (the same trailing "true"/"false" seen after every test suite all session) straight into the shared stream. The fix, suggested directly on seeing the symptom — "Ah I think the child processes would need their own stdio really..." — was to give each spawned child a fully separate, discarded output stream by default, rather than sharing the parent's. Piping the output for later reading was considered and deliberately rejected: an unread pipe fills its buffer and then blocks the child's own writes forever once full, a worse problem than the one being fixed, given no way yet exists to drain one.