Software engineering for systems that keep changing

Software engineering is the practice of making useful changes without losing control of the system. A program that works once is an achievement. A system that remains understandable when its requirements, team, and dependencies change is a different achievement.

Consider an online teaching platform. Initially, a learner joins a course through a single request handler. Later, the product adds limited seats, paid access, waitlists, email notifications, and administrative overrides. The challenge is not finding the right folder names. It is deciding where each rule lives, which changes must happen together, and what the system promises when something fails.

Start with invariants rather than classes

An invariant is a condition that must remain true. For enrollment, useful invariants might be: a learner cannot have two active enrollments in the same course; available seats cannot become negative; and a successful enrollment must remain recorded even if the confirmation email fails.

These statements separate correctness from implementation. They can guide tests, database constraints, and API behavior before any framework is selected. A requirement such as “send a welcome email” sounds simple until you ask whether an email outage should undo enrollment. The invariant makes the answer explicit.

Write down the state transitions as well. An enrollment can move from pending to active or cancelled. A cancellation may release a seat only once. Naming the transitions exposes invalid operations that a collection of unrelated Boolean flags can conceal.

A useful review question is: “Which invalid state does this design make impossible, and which invalid states can still be represented?” A type can eliminate some errors before execution. A database constraint can eliminate others at the persistence boundary. Neither removes the need to reason about concurrent requests.

Put boundaries around reasons to change

A module should own a coherent responsibility. In the teaching platform, enrollment owns seat allocation and enrollment state. Notifications own message delivery. Billing owns payment state. A user interface may coordinate them, but it should not become the sole place where business rules exist.

A practical dependency direction is:

HTTP adapter -> application use case -> domain rules
                       |
                 persistence contract
                       |
               database implementation

The HTTP adapter translates requests and responses. The application use case coordinates an operation. Domain functions express rules that do not require a network connection. A database implementation handles storage details behind an explicit interface.

This structure is useful when it reduces the number of places a rule must be edited. It becomes ceremony when every tiny function gets an interface and a factory without a real need for substitution. Introduce an abstraction when it gives a stable name to a meaningful responsibility or protects a demonstrated boundary.

A modular monolith is a reasonable starting point for many applications. It retains one deployable unit while enforcing ownership inside the codebase. Martin Fowler’s Monolith First discusses the difficulties of discovering service boundaries early. Treat that as an argument to examine, not a universal rule forbidding services.

Make contracts precise enough to be useful

A function returning boolean leaves important questions unanswered. Does false mean the course is full, the user is unauthorized, the database failed, or the request was a duplicate? Distinguish expected business outcomes from unexpected infrastructure failures.

type EnrollmentResult =
  | { status: "enrolled"; enrollmentId: string }
  | { status: "already-enrolled"; enrollmentId: string }
  | { status: "full" }
  | { status: "not-eligible"; reason: string };

function canEnroll(input: {
  prerequisitesMet: boolean;
  enrollmentOpen: boolean;
}): { allowed: true } | { allowed: false; reason: string } {
  if (!input.enrollmentOpen) return { allowed: false, reason: "Enrollment is closed" };
  if (!input.prerequisitesMet) return { allowed: false, reason: "Prerequisites are incomplete" };
  return { allowed: true };
}

This pure function makes a rule easy to understand and test. It does not reserve a seat. That distinction matters: a successful eligibility check is not proof that a seat remains available when the write happens.

Contracts also include time and failure semantics. State whether an operation is idempotent, whether a timeout means “not executed” or “outcome unknown,” and whether a successful response guarantees durable storage. Specify versioning behavior for external consumers before changing a field’s meaning.

Correctness must survive concurrent requests

Suppose two requests both read one available seat. Each passes an application-level check and each creates an enrollment. A correct sequential function has produced an incorrect concurrent system.

Use the database to enforce the invariant atomically. An illustrative enrollment transaction can insert a unique learner-course enrollment, decrement capacity only when a seat is available, and roll back if either step cannot complete. The exact locking and conflict strategy depends on the schema and database, but the check and the write must not be separated by an unprotected race.

Useful schema protections include a unique constraint on the learner-course relationship and a check constraint preventing negative capacity. PostgreSQL’s constraint documentation describes these mechanisms. Constraints defend stored state even if a new import script or administrative endpoint bypasses one application code path.

Do not hold a database transaction open while waiting for an email provider. Long transactions retain locks and make an external dependency part of your contention problem. Record a notification job alongside the enrollment in the same local transaction, then let a worker deliver it. This is the motivation for a transactional outbox.

The worker may receive the same job more than once. Give the notification a stable identity and make repeated processing safe. An outbox reduces one class of lost-event problem; it does not magically provide exactly-once behavior across unrelated systems.

Test at the boundary where the risk exists

Use different tests for different questions. A unit test checks a rule in isolation. An integration test checks whether the database really enforces uniqueness and transaction behavior. An API test checks the request contract. An end-to-end test checks whether a learner can complete the journey through the browser.

Testing the same condition at every layer can create a large, slow suite without proportionate confidence. Put detailed edge cases near the rule, and use a smaller number of broader tests to verify that the pieces are connected correctly.

For enrollment, important cases include a repeated request with the same idempotency key, two users competing for the last seat, cancellation delivered twice, and a notification failure after enrollment commits. The concurrency case needs a real concurrent integration test; mocking two sequential calls does not reproduce the race.

A useful property is that available seats never become negative under any supported sequence of operations. Another is that replaying a successful idempotent request does not change the final state. Property-based and state-machine tests can explore these patterns more broadly than a few hand-picked examples.

Change data and APIs in compatible steps

A database migration is a deployment, not just a file. If an old application instance and a new instance run simultaneously, both may need to read the same database. A column rename that works on a developer laptop can break half the fleet during a rolling deployment.

Use an expand-and-contract sequence when compatibility requires it. Add the new representation, deploy code that tolerates both, backfill in controlled batches, verify completeness, switch reads, and remove the old representation only after old consumers are gone. Design rollback around the actual data changes, not merely the ability to redeploy an old binary.

For public APIs, adding an optional field is often easier to roll out than changing the meaning of an existing one. Version contracts where necessary, document deprecation windows, and observe real consumer usage before removal. A type checker can help callers inside one repository; it cannot update every independent integration.

Make failures diagnosable

A system is easier to maintain when it explains its own behavior. Assign a request or operation identifier that can connect the HTTP request, transaction, job, and external delivery attempt. Log state transitions and error categories instead of indiscriminately recording complete request bodies.

Track user-relevant signals: enrollment success rate, capacity conflicts, queue age, and notification delay. A server returning 200 for every request may still fail the user if it silently drops jobs. Conversely, a rising “course full” rate may be an expected business outcome rather than an infrastructure incident.

Set dependency deadlines and make retries intentional. A retry can duplicate an operation whose response was lost, so its safety depends on the operation’s contract. The AWS Builders’ Library article on timeouts, retries, backoff, and jitter explains why retry behavior belongs in system design rather than being sprinkled into exception handlers.

Keep changes small enough to reason about

A good change description states the problem, the new behavior, and how it was checked. “Refactor enrollment” is hard to assess. “Repeated enrollment requests now return the existing enrollment without consuming another seat” gives a reviewer a concrete contract.

Separate an invariant-preserving refactor from a change in business behavior when doing so makes review easier. Avoid scattering a feature across many unrelated abstractions simply to make each file small. The unit of understanding is the operation and its consequences, not the number of lines in a class.

Record consequential architectural decisions briefly: the context, the options considered, the choice, and conditions that would justify revisiting it. “We use one database transaction because seat allocation and enrollment must commit together” is more durable than a diagram containing only product logos.

Practice: evolve a feature without losing its guarantees

Implement enrollment with a documented state machine, unique enrollment records, and an atomic capacity update. Add tests for the last-seat race and repeated requests. Then introduce asynchronous email delivery and deliberately make the email provider fail.

The enrollment should remain durable, the notification should be retryable, and the trace should explain both outcomes. Finally, add a waitlist without rewriting the HTTP layer, domain rules, and database model indiscriminately. Notice which boundaries made the change easier and which abstractions created work without protecting a useful contract.

Good software engineering makes future reasoning cheaper. The result is not a system that never changes. It is one whose important guarantees remain visible while change happens.