EFL-AN-2025-001 / ARCHITECTURE NOTE
Abstract
Many applications begin with a straightforward execution model. A request reaches the application, a controller or handler validates it, domain logic runs, data is persisted and a response returns to the caller. For a large class of software this model is sufficient, and adding another execution layer would create complexity without solving a real problem.
Our systems began to depart from that model as more operations crossed boundaries we did not control. A single user action could require local validation, document generation, interaction with an external authority, cryptographic processing, transmission, persistence of intermediate state and later recovery if one of those steps produced an inconclusive result. In several cases the HTTP request ended before the operational question had been completely resolved.
We gradually moved responsibility for that work into an application-owned runtime. We use runtime here in a deliberately narrow sense: an execution environment inside the application architecture that coordinates operational work independently of the interface through which that work was initiated. It is not a virtual machine, an operating system or a replacement for the underlying programming framework.
This note describes why we introduced that boundary, what changed after doing so and, equally important, what we decided should remain outside it.
1. The request lifecycle worked until the operation outgrew it
Our earlier application flows looked conventional:
Request ↓ Controller ↓ Service ↓ Database ↓ Response
There is nothing inherently wrong with this architecture. In fact, keeping execution this simple is preferable while the work being performed fits naturally inside the request that initiated it.
The first difficulties appeared when user actions began producing operations whose lifecycle extended beyond that request. Consider a document that must be assembled locally, validated, passed through an external process and transmitted to another system that ultimately determines its acceptance. Several meaningful states can exist between the initial click and the final external result.
A controller can certainly execute all of those steps. We did this in early implementations. The problem became visible when failures stopped aligning neatly with the beginning and end of the HTTP exchange.
A remote service could receive an operation while its response never returned. A process could complete one external step and fail before recording the result of that step locally. An unavailable dependency might prevent one part of the workflow while leaving the rest of the application perfectly usable. We also had operations that later needed to be resumed without repeating work already completed.
At that point, the request was still useful as the mechanism that started the work, but it was becoming a poor representation of the work itself.
2. We found lifecycle assumptions in unexpected places
Moving code out of a controller is easy. Removing the assumptions that accumulated around the controller is considerably harder.
We noticed this when operations originally available only through the user interface later needed to run from scheduled processes, administrative recovery procedures or other system interfaces. Business logic could be extracted into a service, but that service often still assumed things inherited from its original environment: an authenticated web request existed, errors could be converted directly into an HTTP response, temporary state could remain in memory, or the caller would still be present when processing finished.
Those assumptions had not looked architectural when they were introduced. They were simply convenient properties of the first implementation. Once the same operation had several possible entry points, however, the coupling became much easier to see.
Web interface ─────┐
│
Scheduled work ────┤
├──► Application boundary ──► Operational work
System interface ──┤
│
Recovery process ──┘
We wanted each entry point to describe what was being requested without also becoming responsible for defining how the complete operation lived, failed and recovered. This was the practical reason for introducing a separate runtime boundary.
3. What we mean by an application-owned runtime
The term runtime is broad enough to cause confusion, so our use of it needs qualification. We are not referring to PHP, a JVM, a JavaScript engine or another language execution environment. Nor are we attempting to reproduce capabilities already provided reliably by the operating system, database or web framework.
In our architecture, the runtime owns the execution context for operational work after that work has crossed the application boundary.
Input ↓ Application Boundary ↓ Operational Runtime ↓ Outcome
Inside that boundary there may be validation, state transitions, interaction with external components, persistence and recovery. The exact sequence varies by system, and we deliberately avoid making every operation pass through machinery it does not need.
What changed for us was where responsibility lived. A web controller could initiate work without having to remain responsible for its complete lifecycle. A scheduled task could initiate the same kind of work without recreating rules that previously lived in the controller, while recovery could re-enter an existing operation through an explicit mechanism rather than by pretending that the original request happened again. That last case became particularly important once external effects were involved.
4. An HTTP response is not an operational result
One of the clearest reasons for separating these concepts appeared in integrations with external systems.
Suppose an application sends a document to an external authority. The remote service processes it successfully, but the connection disappears before its response reaches the caller. From the web layer, the request may end as an error. From the external system’s perspective, the operation may already be complete. The reverse can happen as well: a perfectly successful HTTP exchange can contain an application-level rejection.
If operational state is derived directly from transport state, those distinctions become difficult to preserve. 200 starts to mean success, exceptions start to mean failure, and retry logic grows around classifications that were never intended to describe the business operation.
This problem eventually influenced our state modeling more than the networking code itself. We needed somewhere to preserve what had actually been established about an operation even after the interface that initiated it had already returned, and that responsibility gradually moved into the runtime.
Information arriving from the transport layer can then be incorporated without requiring transport terminology to become the vocabulary of the operation. The web interface remains responsible for telling the user what can reasonably be said at that moment; the operational record can remain unresolved when the available information does not yet support a stronger conclusion.
5. Persistence changed meaning once operations could survive requests
When all relevant work happens inside one request, temporary execution state can disappear harmlessly after the response. Longer-lived operations require a different approach.
We began identifying the minimum information needed to continue reasoning about an operation after its original process no longer existed. Depending on the system, that might include a local operation identifier, validated input, an external correlation identifier, the last confirmed state, timestamps and enough information about an external interaction to determine what recovery remains safe.
We did not find it useful to persist every intermediate object. Doing so made the runtime harder to understand and increased the number of representations that could later disagree. Instead, persistence was designed around continuation: if the process stops here, what must still exist for another execution to determine where the operation was and what may safely happen next?
That question produced considerably smaller records than simply serializing the execution context. It also changed how we looked at logs. Logs remained valuable for diagnosis, although we stopped expecting a log stream to provide the authoritative representation of unfinished operational work. A system that needs an engineer to reconstruct its current state from yesterday’s log files has preserved useful evidence for humans without necessarily preserving enough state for the software itself to continue.
6. Recovery should enter through the architecture
Early recovery procedures in small systems are often pragmatic. An engineer fixes a value, reruns a script or invokes the same endpoint again. There is nothing unusual about that during development. Difficulties appear when those procedures become normal production behavior.
If recovery requires bypassing application boundaries, manually reproducing an HTTP request or changing database state without passing through the same invariants used during ordinary execution, the system effectively acquires a second architecture that appears mainly when something goes wrong.
We wanted recovery to become an expected entry point. Some failures still require human investigation, particularly when an external system provides insufficient information to determine an outcome, but the runtime gives us a consistent place from which the operation can be inspected and, once the next safe action is known, continued.
Recovery procedures also exposed assumptions in normal execution that ordinary success paths had hidden. When an operation could not be resumed without recreating an entire web request, we usually found that some of its boundaries were still tied too closely to the original interface.
7. The runtime had to become smaller as it became more important
Our first instinct when establishing a central execution layer was to move responsibilities into it. That works for a while and then creates another problem: the runtime begins learning too much about every system connected to it.
A fiscal integration has terminology and rules that do not belong in a generic execution mechanism. REST clients, XML serializers, database connectors and file readers also introduce technology-specific concerns. If all of them become runtime concepts, adding each new integration expands the runtime’s knowledge until it becomes the most coupled component in the architecture.
We gradually moved in the opposite direction. The runtime should understand the contracts required to coordinate operational work, while specialized interpretation remains closer to the domain that owns it and communication mechanics remain closer to the technology performing them.
This boundary is still something we refine. There are cases where validation, identity or error classification can reasonably exist at more than one layer, and forcing every concern into a theoretically perfect location usually creates more ceremony than clarity.
A practical test has been whether replacing an external technology changes the runtime’s understanding of the operation. When changing REST to another transport forces us to redesign operational concepts that have nothing to do with transport, we usually find that technological detail has crossed farther into the architecture than intended.
That observation eventually led to a separate piece of architecture: the adapter layer.
8. Runtime ownership does not mean runtime isolation
Separating operational execution from the application lifecycle could easily produce an isolated subsystem with its own model of everything. We wanted to avoid that.
The application still owns authentication, user interaction, configuration and the domain capabilities exposed to users. Databases continue to provide durable storage. External services remain authoritative for the things they control. Queues, schedulers and transport clients retain the responsibilities appropriate to those technologies. The runtime coordinates work across those boundaries without attempting to replace them.
This has kept us from treating the runtime as a universal solution. A simple read operation does not become better merely because it passes through another abstraction. A form submission whose effects are completely local may be clearer as ordinary application code.
We introduce runtime-managed execution where the operation has enough independent lifecycle, external consequence or recovery requirement to justify it. We also found that this boundary carries a cost: additional contracts, states and tests, together with another architectural concept that developers need to understand. We accept that cost where it removes greater ambiguity elsewhere.
9. The application became easier to change around it
One consequence appeared gradually rather than as an original objective. Once operational work no longer belonged to the web controller, changing the interface became less consequential. A workflow could begin in a browser, a PWA, an administrative tool or another system without requiring each interface to reproduce the complete execution logic.
The same happened with external dependencies. Moving technology-specific behavior behind explicit boundaries made it easier to replace or supplement an integration without rewriting the operation that depended on it.
This does not make the architecture technology-independent in an absolute sense. Real systems always inherit constraints from their databases, protocols and deployment environments. What improved was our ability to locate those constraints. In practice, changes became easier to reason about because we could usually identify whether they originated in the operation itself or in one of the technologies surrounding it.
10. Where we currently use the boundary
We now tend to consider a dedicated operational runtime when several conditions appear together: the operation crosses external boundaries, can outlive its initiating request, has intermediate state worth preserving, may require reconciliation or recovery, or can be initiated from more than one interface.
None of those conditions individually mandates a runtime. A long-running job can be handled perfectly well by a queue worker. A remote API can be called directly from an application service. A scheduled task does not need a special execution model merely because cron initiated it.
The boundary becomes useful when the lifecycle of the operation itself has become something the application needs to reason about.
We missed that distinction in some of our earlier implementations. Execution was initially organized around the mechanisms available to us—controllers, jobs, commands and API clients—and later we found that one logical operation could move through several of them. Giving the operation an execution context of its own allowed the surrounding mechanisms to return to narrower roles as ways of entering, transporting or continuing work.
Closing observation
The decision to introduce our own runtime did not begin as an attempt to build infrastructure. It emerged from fairly ordinary production problems: requests ending before operations were resolved, external systems returning incomplete evidence, recovery procedures accumulating beside normal execution and the same work gradually acquiring more than one entry point.
Separating operational lifecycle from request lifecycle gave those problems a common architectural location. It also created a constraint we did not appreciate immediately: if the runtime was going to remain reusable across systems, it could not absorb the semantics and technologies of every integration connected to it.
That unresolved boundary became the starting point for the next architecture problem we worked on—how to connect a domain-neutral runtime to very different sources, destinations and technologies without teaching the runtime what each of them is.
Document record
Document ID: EFL-AN-2025-001
Title: Why We Give Operational Systems Their Own Runtime
Document type: Architecture Note
Category: Architecture Notes
Publication year: 2025
Institution: EventFlow Labs
Language: English
Revision: 1.0