EFL-ES-2025-003 / ENGINEERING STANDARD
Separating what can be validated syntactically from what requires domain meaning and what can authorize an operation.
Abstract
Validation is often treated as a single step, but in operational systems we repeatedly found different kinds of checks being grouped under the same label. A payload could be structurally correct and still represent an invalid domain value. A record could be semantically valid and still be insufficient for an operation whose consequences required fresher or stronger evidence.
This became especially visible at system boundaries. Technology bindings could confirm that incoming data had the expected shape, while domain connectors were better positioned to determine whether identifiers, statuses or classifications meant what the contract expected. Later in the operation, the system still needed to decide whether the available information justified proceeding with an external effect.
We therefore began separating validation into three practical concerns: structural validity, semantic validity and operational sufficiency. The separation is not a rigid pipeline and the same information may be checked more than once as additional context becomes available. Its value is that a component does not need to make a stronger claim than the evidence available at its own boundary supports.
This standard documents how we currently use those distinctions, where they overlap and how they affect error representation, recovery and conformance testing.
1. A valid payload can still be wrong
Consider a simple input:
{
"identifier": "0012345",
"amount": 150000,
"currency": "PYG"
}
A boundary can establish several useful facts immediately. identifier is a string, amount is numeric and positive, and currency belongs to the set of values accepted by the contract.
Those checks matter. They prevent malformed information from moving deeper into the system and allow errors to be reported close to their source.
They do not establish that 0012345 identifies an entity known to the relevant domain. They do not tell us whether the entity is currently eligible for the operation or whether the amount is coherent with a business rule that depends on information not present in the payload.
We encountered bugs where these distinctions had been collapsed because the input had already passed something named validate(). Downstream code treated that result as stronger evidence than the validator had actually produced.
The validator itself was not necessarily wrong. The problem was that nobody had stated what kind of validity it established.
2. Structural validation belongs close to representation
Structural checks concern the form required by a contract.
Depending on the boundary, they may include required fields, primitive types, lengths, enumerated values, encoding, date formats or the shape of nested structures.
A simplified structural contract might require:
identifier:
type: string
min_length: 1
amount:
type: number
greater_than: 0
currency:
enum:
- PYG
- USD
These rules can usually be evaluated without knowing much about the wider domain, which makes them suitable for boundaries where an external representation first enters the system. An HTTP binding can reject malformed JSON, a file importer can detect a missing column, and an XML parser can establish whether required elements exist before domain interpretation begins.
We avoid placing domain rules here merely because the data is already available. A technology binding that knows how to parse a fiscal identifier does not automatically become the right place to determine whether that identifier exists or whether the entity it represents may participate in a particular operation.
Keeping structural validation relatively narrow has also made bindings easier to reuse.
3. Structural validity can preserve invalid meaning
Some defects survive every schema check.
Suppose an external dataset contains:
identifier = "0012345" status = "PENDING"
The structure may be completely valid. The harder question is what PENDING means.
One external system may use it after accepting a request for asynchronous processing. Another may use the same label before the request has crossed any authoritative boundary. Mapping both directly into the same internal state can create a semantically invalid result without violating a schema.
Identifiers produce similar cases. A string may satisfy every length and character rule while referring to no entity at all. Two identifiers with different formatting may represent the same entity, or two visually similar values may represent different identifier types.
Those questions require knowledge about the domain being translated.
In our adapter architecture, much of that work belongs in the Domain Connector because that is where external representation becomes an internally meaningful contract. The exact location can vary, but the semantic rule needs an owner that understands what the value represents.
4. Semantic validation often requires another source of information
Not every semantic rule can be evaluated from the submitted value itself.
A checksum may establish that an identifier is internally plausible. Confirming that the identifier currently belongs to an existing entity can require an authoritative registry. Determining whether a classification remains current may require a synchronized reference dataset or a live query.
This means semantic validation can have availability and freshness characteristics of its own.
We encountered this while working with externally maintained reference information. A locally synchronized record could still represent a valid entity even when the authority was temporarily unreachable. Whether that record was sufficiently current depended on what the application intended to do with it.
For a descriptive lookup, yesterday’s valid representation may be acceptable. For a decision that depends on current regulatory status, the same information may no longer be sufficient.
The semantic value has not necessarily become false; what changed is the evidence available about whether it remains current enough for the next operation. In practice, this meant we could no longer use semantic validity by itself as permission to proceed.
5. Operational validation asks a different question
An operation can receive structurally valid input and semantically valid domain information and still be unable to proceed safely.
Imagine that an external submission previously timed out after transmission. The local record is valid, the destination is known and the payload itself passes every domain rule. The unresolved issue is whether performing the operation again could create a duplicate external effect.
Nothing about the structure of the payload can answer that.
At this point the relevant check concerns the current operational state and the consequences of the next action.
A simplified decision might inspect:
input_valid = true domain_valid = true previous_outcome = UNRESOLVED idempotent_retry_available = false
The appropriate result may be to withhold another submission until reconciliation occurs.
We refer to this class of decision as operational validation because it evaluates whether the conditions currently known to the system are sufficient to permit the requested transition or effect.
6. Valid data can become insufficient data
Freshness made this distinction particularly useful.
Suppose a customer record was synchronized from an authoritative registry three days ago. Its identifier is valid. The name and classification were valid when retrieved. Nothing in the local representation is structurally damaged.
Now an operation requires confirmation of a status that may have changed since the synchronization.
The system should not need to relabel the local record as semantically invalid merely because it cannot establish the present value with enough confidence for this operation.
We found it clearer to preserve the valid historical representation and evaluate sufficiency separately.
Conceptually:
record_valid = true record_age = 72 hours operation_requires_current_status = true authority_available = false operational_result = INSUFFICIENT_EVIDENCE
Another operation using the same record might continue because it depends only on stable descriptive data. In practice, we can then apply a stricter freshness requirement to one operation without changing how the underlying record is represented elsewhere.
7. Validation can occur more than once
A rigid validation pipeline sounds attractive:
STRUCTURE ↓ SEMANTICS ↓ OPERATION
Real operations are less tidy.
Input may pass structural validation when first received, then acquire additional fields from an external lookup. Those fields need validation too. A semantic rule may be evaluated once from cached data and again later after fresh authoritative information becomes available. Before an external effect is produced, operational state may have changed since the original request began.
In implementation, the three categories have worked better for us as descriptions of the questions being asked. They do not imply that each check occurs once or in a fixed order.
A document can be structurally checked before persistence and again after serialization. An identifier can be semantically validated when entered and later rechecked if the operation requires current authority. Operational validation can occur immediately before an external effect even though similar checks were performed earlier in the workflow.
Repeated validation is useful when the evidence or state changed. Repeating the same expensive check mechanically at every layer usually is not.
8. The layer performing a check should have the evidence it needs
Some validation problems were caused by placing rules where the required information was unavailable.
A technology binding knows a great deal about representation and transport. It may know that an HTTP response contains valid JSON, that a required field exists and that a certificate exchange succeeded. It often lacks the domain context necessary to decide what the returned identifier means.
The opposite mistake is possible as well. A runtime coordinating an operation should not have to parse XML namespaces or know which CSV column contains an external code simply so that it can evaluate a domain condition.
We use the adapter layers to keep much of this knowledge near the evidence that supports it:
Technology Binding
representation and transport evidence
Domain Connector
domain interpretation and semantic evidence
Operational boundary
current state and permission to continue
The boundaries are not absolute. Authentication is one example where transport evidence can carry domain significance, and some validation rules genuinely need information from more than one layer. In those cases the relevant evidence can travel with the operation instead of requiring each layer to reconstruct the same interpretation.
9. Collapsing every failure into INVALID made recovery harder
Earlier implementations sometimes exposed validation as a binary result:
VALID INVALID
That works when the only question is whether an input can be accepted.
It became inadequate once validation participated in recovery and operational decisions.
Consider four cases:
malformed identifier well-formed identifier not known to domain valid record but authoritative status unavailable valid operation currently blocked by unresolved prior outcome
Calling all four INVALID loses information that determines what can happen next.
A malformed identifier should normally be corrected by the caller. A domain lookup failure may justify trying another authoritative source. Insufficient evidence may resolve automatically when connectivity returns. An unresolved prior effect may require reconciliation rather than input correction.
The exact vocabulary varies between contracts, but we commonly preserve distinctions equivalent to:
INVALID_STRUCTURE INVALID_SEMANTICS INSUFFICIENT_EVIDENCE OPERATION_NOT_PERMITTED
We do not require every system to expose these exact codes. Conditions requiring different recovery behavior should not be forced into one status merely because they all prevented the current operation from continuing.
10. Validation errors also need conformance
Once multiple implementations perform the same contract, success-path validation is only part of the conformance question.
Suppose a remote registry implementation returns NOT_FOUND for an identifier whose authoritative source explicitly confirms absence. A local dataset implementation lacks the identifier because its synchronized dataset is incomplete.
If both implementations map that condition to the same semantic validation error, consumers may make assumptions the local implementation cannot justify.
We therefore include validation outcomes in contract-level conformance cases where the distinction matters.
A vector may specify:
condition:
authoritative absence confirmed
expected:
semantic_result: INVALID
reason: NOT_FOUND
while another case specifies:
condition:
authoritative status unavailable
expected:
semantic_result: UNRESOLVED
The remote and local implementations may arrive at these outcomes through completely different mechanisms, which is acceptable for the conformance case. What the consumer observes is the conclusion each implementation is prepared to support at the contract boundary.
This is where the previous standards on contract conformance and reproducible vectors became directly useful. Validation rules that affect observable behavior can be tested independently of the component that implements them.
11. We avoid validation rules whose consequence is unclear
A surprisingly common pattern in mature systems is a validation rule that nobody can explain operationally.
A field is rejected because “it has always been required.” A format is normalized because an old integration expected it. A status blocks an operation even though the external system no longer uses that status in the same way.
We now ask what consequence a validation rule protects before promoting it into a shared contract.
For structural rules this may be simple: the downstream parser cannot operate without the field.
A semantic rule may protect identity or prevent information from being associated with the wrong domain entity.
Operational rules usually protect a transition, an external effect or an invariant that would be expensive to repair after the fact.
If nobody can identify the consequence, we treat the rule cautiously. It may still be correct, but inherited validation is a poor basis for a standard.
This review has removed some rules and moved others to narrower contexts where their purpose was clearer.
12. Strictness depends on where the consequence occurs
The same data can justify different validation policies in different operations.
A user interface that searches a local reference dataset can often tolerate partial descriptive information. A process preparing an externally regulated document may need stricter requirements. A diagnostic interface may deliberately expose malformed upstream data because hiding it would make investigation harder.
We therefore avoid defining “strict validation” as a global platform property. The consequence of accepting the information depends on the operation.
If the next step only displays a tentative lookup result, rejecting every incomplete record may reduce usefulness without reducing meaningful risk. If the next step creates an external financial or regulatory consequence, the threshold can be much higher.
This is also why validation policy belongs in contract documentation rather than being scattered exclusively through helper methods. Developers need to know which operation a rule protects when deciding whether it can safely change.
13. Evidence can improve after validation fails
Some validation failures are final for the submitted input. Others describe the information currently available.
A malformed identifier will remain malformed until somebody changes it. An INSUFFICIENT_EVIDENCE result may change without changing the original input at all.
For example, a registry may be temporarily unavailable:
identifier = structurally valid local_reference = absent authority = unavailable
The system cannot establish the semantic result it needs.
Later:
authority = available authoritative_record = found
The same input can now pass the semantic check.
This difference affects persistence and recovery. We do not want a temporary lack of evidence stored permanently as though the data had been proven invalid.
When a validation outcome can change solely because new evidence becomes available, retaining that distinction gives recovery logic somewhere to resume.
14. What we currently expect from a boundary validator
We do not require every validator to implement all three kinds of validation.
A structural validator can legitimately stop after checking representation. A domain validator may assume that structural checks already occurred. An operational guard can consume previously validated objects and focus only on whether the current transition is permitted.
What we expect is that the responsibility is explicit.
For a validation component used at a shared boundary, we normally want to know:
what evidence it receives what claim it establishes which failures it distinguishes whether the result can change without changing the input what subsequent operation is allowed to rely on that result
Those questions have been more useful in reviews than asking whether a component is simply “the validator.”
They also give us something concrete to test. If a validator claims only structural validity, a semantic conformance suite should not be attached to it. If an operational guard can return INSUFFICIENT_EVIDENCE, the recovery behavior associated with that outcome belongs in the surrounding contract.
Closing observation
Some of the hardest validation defects we encountered were produced by code that was technically doing exactly what it had been written to do.
The problem was the strength of the conclusion attached to its result. A structural check returned true, and another layer treated that as domain validity. A valid domain record was considered sufficient for an operation whose required evidence had expired. A temporary inability to verify something externally was persisted as though invalidity had been established.
Separating these checks gave us a more precise vocabulary for reviewing those cases. It also reduced the pressure to create one validator capable of knowing everything about the input, the domain and the current operational state.
The same information can acquire stronger validation as more evidence becomes available. We now try to make that progression explicit before the system is allowed to turn the information into a consequence that will be harder to reverse.
Document record
Document ID: EFL-ES-2025-003
Title: Validation at System Boundaries: Structure, Semantics and Operational Consequence
Document type: Engineering Standard
Category: Engineering Standards
Publication year: 2025
Institution: EventFlow Labs
Language: English
Revision: 1.0