Let Each Part Do Its Job

Guide 3 of 5 · Architecture, messaging, and design patterns

Series overview · Previous: Networking and APIs · Next: Reliability, security, and operations

1. A school has responsibilities, not just rooms

A school has a library desk, a catalog, a notification process, and people who manage each. Some responsibilities can share a room. Others need a separate workspace.

Software has the same distinction. A layer separates responsibilities in code. A tier is a deployment boundary. Three code layers do not require three machines.

Presentation handles user interaction. Business logic enforces rules. Data access reads and writes storage. In closed layering, a layer calls the next layer down. Open layering permits some layers to be skipped. Choose boundaries that clarify dependencies instead of adding empty pass-through steps.

2. Monolith or microservices?

A monolith is deployed as one application unit. It can still run many instances behind a load balancer.

A modular monolith has clear internal boundaries. The catalog module does not casually edit reservation internals. It is like departments sharing one school building while keeping responsibilities clear.

Microservices split capabilities into independently deployable services. Each service owns its rules and data access. Ownership does not require one dedicated physical database server per service; it does require a clear boundary over who can change the data.

Choice	Useful advantage	New or remaining cost
Monolith	Straightforward deployment and local calls	Whole application often deploys together
Modular monolith	Internal separation without many network calls	Boundaries need discipline
Microservices	Independent deployment, scaling, and ownership	Network failures, operations, contracts, distributed workflows

Move a capability out when there is a reason: a different scaling need, separate release ownership, or a boundary that is already understood. A small team does not gain autonomy by maintaining 30 services nobody has time to operate.

A distributed monolith has separate services but still needs coordinated releases or tightly coupled calls for ordinary work. The network costs remain while much of the independence disappears.

SOA, or service-oriented architecture, also organizes capabilities behind service interfaces. An enterprise service bus can centralize routing and transformation for integrations. That can help when many legacy systems speak different formats, but central ownership and shared failure paths deserve attention. A message broker does not automatically provide all the transformations of an ESB.

3. Synchronous and asynchronous work

If a librarian waits at the printer until a large report finishes, nobody else gets help. Instead, the librarian can accept the request, give it a job number, and return to the desk.

Synchronous work requires a caller to wait for the result in that interaction. Asynchronous work can finish later. It improves responsiveness only when the product can honestly separate acceptance from completion.

Search normally needs an immediate answer. A monthly borrowing report can run in the background. A reservation must not be called confirmed until its required state change succeeds.

flowchart TD
    U[Student requests report] --> A[API validates request]
    A --> J[(Durable job record)]
    J --> P[Dispatcher retries pending jobs]
    P --> Q[Job queue]
    Q --> W[Report worker]
    W --> O[(Private report storage)]
    W --> J
    U --> S[Check job status]
    S --> J

The durable job record lets a dispatcher recover from a crash before enqueueing. A transactional outbox is another way to make that handoff reliable. The worker updates status and stores the result; the API returns 202 when it has accepted durable work.

4. Queue, pub/sub, and stream

A message is data sent between components. A command asks for an action, such as GenerateReport. An event states a fact, such as ReservationCreated.

Model	School example	What matters
Work queue	Several assistants share report jobs	One successful processing outcome per job is intended; redelivery can happen
Publish-subscribe	Catalog update informs search and notifications	Independent subscriptions receive relevant events
Retained stream	A dated history of catalog changes	Consumers track positions and may replay retained records

Pub/sub does not inherently mean push-only or no storage. Delivery, retention, ordering, and replay depend on the implementation.

A queue smooths bursts. It does not create processing capacity. If 100 jobs/s arrive and workers complete 80 jobs/s, backlog grows by about 20 jobs each second. In ten minutes, that is 12,000 extra jobs unless the rates change.

Backpressure limits incoming work or slows producers when downstream capacity is exhausted. Bounded queues, quotas, and honest rejection are often better than accepting work that will miss its deadline by hours.

5. Delivery is different from business effect

Suppose a worker creates a report and crashes before telling the queue it finished. The queue can deliver the job again.

Term	Meaning	Risk
At-most-once	Avoid redelivery	Work can be lost after some failures
At-least-once	Retry until acknowledged under the system's guarantees	Duplicate processing is possible
Exactly-once effect within a defined boundary	A specified result is committed once	Requires suitable coordination, transactions, or deduplication

FIFO means first-in, first-out ordering within the system's stated scope. It does not, by itself, make every external action happen exactly once.

For a database effect, an idempotent consumer can write the effect and a processed-event marker in the same transaction. A unique key on the event ID prevents two workers from committing the same effect. A crash after commit but before acknowledgment leads to redelivery, which finds the marker and safely skips the duplicate.

If the effect happens in an external service, a local marker alone is not enough. A crash can occur between the external effect and the marker. Use that service's idempotency support, reconciliation, or another explicit protocol.

6. Kafka vocabulary without magic

In Kafka, records are stored in topic partitions. Within a partition, an offset identifies a record's position. A consumer group tracks its progress; different groups can read the same retained data independently. Ordering is per partition, not a global order across all partitions.

Kafka supports transactions and exactly-once processing within supported boundaries. Writing results to an arbitrary external system still needs coordination with that system. These are configuration-sensitive guarantees, not properties to assume from a product name. Kafka design documentation

For a catalog change stream, use a stable book ID as the partition key if per-book order matters. A popular key can concentrate work on one partition. Changing partition count can affect key mapping, so plan ordering during migration.

Use a broker with task-oriented routing when the problem is distributing jobs. Consider a retained log when replay and independent consumers are central. Evaluate actual features and workload; avoid a blanket "Kafka is better than queues" rule.

7. Retries need limits and a destination

A failed message can be retried after a delay. Exponential backoff increases delays between attempts; jitter adds variation so workers do not all retry together.

A dead-letter queue stores messages that exceed the retry policy or need investigation. It requires an owner, an alert, an explanation of the failure, and a safe replay process.

Do not confuse a poison message that repeatedly fails with a deliberately designed shutdown sentinel. Their handling is different.

For ordered processing, decide what happens when one event fails. Skipping it may violate later operations. You may pause that key, repair the event, or use a business-specific recovery path.

8. Outbox: do not lose the announcement

We want a reservation row and a ReservationCreated event. Writing to the database and then separately publishing can fail halfway.

Write the reservation and an outbox row in the same local database transaction. A relay later publishes committed outbox rows. This preserves the obligation to publish even if the app crashes. The relay can publish twice, so consumers still need duplicate protection. AWS transactional outbox guidance

sequenceDiagram
    participant A as Reservation app
    participant D as Database
    participant R as Outbox relay
    participant Q as Broker
    participant W as Consumer
    A->>D: Begin transaction
    A->>D: Save reservation and outbox event
    A->>D: Commit both
    D-->>A: Success
    R->>D: Read committed pending events
    R->>Q: Publish event with stable ID
    Q-->>R: Publish acknowledgment
    R->>D: Mark event published
    Q->>W: Deliver, possibly again after failure

If publication succeeds but the relay crashes before marking it, the next attempt may duplicate the event. This is a deliberate, manageable failure mode compared with losing the event silently.

9. Saga: complete a workflow across services

A saga coordinates several local transactions. If a later step fails, the system performs business actions that compensate for earlier steps.

Imagine arranging a school event: reserve a room, reserve equipment, and assign a helper. If no equipment is available, release the room. Releasing it is a new action; it does not erase the fact that the reservation happened.

flowchart TD
    S[Start event booking] --> R[Reserve room]
    R --> E{Equipment available?}
    E -->|Yes| H{Helper assigned?}
    E -->|No| U[Release room]
    H -->|Yes| C[Confirm booking]
    H -->|No| X[Release equipment]
    X --> U
    U --> F[Mark booking unsuccessful]

An orchestrated saga has a coordinator tracking steps. Choreography lets services react to events. The first makes the workflow easier to see centrally; the second can reduce direct coupling but make long workflows harder to follow.

Compensation can fail too. Persist workflow state, make steps repeatable, and provide reconciliation. A saga does not supply the isolation of one ACID transaction across every service.

10. Event-driven, event-sourced, and CQRS are different

These ideas can work together, but none requires the other two.

Idea	Question it answers	Library example
Event-driven architecture	How do components react to changes?	Search updates after BookAdded
Event sourcing	What is the source of truth?	Store reservation events and rebuild state from them
CQRS	How are write and read responsibilities organized?	Reservation commands and a separate availability read model

With event sourcing, an append-only event history is authoritative. A projection turns events into a useful current view. Snapshots can reduce replay work. Plan event versioning, replay safety, and how sensitive data is handled throughout its lifetime.

CQRS separates command and query models. It can use one database; separate databases and asynchronous projections are optional. A delayed projection needs a clear user experience after a successful command. CQRS pattern

For a small library, ordinary current-state tables may be simpler. Add event sourcing for a real need for domain history and reconstruction, not just because an audit log sounds useful.

11. Object design patterns: small tools inside the bigger system

Architecture patterns arrange services and data. Object design patterns arrange objects and their behavior. A circuit breaker and a Builder operate at different levels even though both are called patterns.

A pattern is a reusable idea, not a library you install. Start with the problem. The tables below use new school and document examples rather than source-kit code.

Creating objects

Pattern	Simple picture	Software use	Watch for
Simple Factory	A stationery desk picks the right notebook	One creation function selects a type	A large switch can become hard to maintain
Factory Method	Each workshop chooses its own tool	A subclass overrides a creation step	Inheritance can add complexity
Abstract Factory	Get a matching set of classroom supplies	Create compatible families of related objects	Adding a new product kind affects factories
Builder	Fill a trip form one choice at a time	Construct a complex valid object in steps	Reject invalid combinations at build time
Prototype	Copy a lesson-plan template	Clone an existing configured object	Shallow copies can share mutable data
Singleton	One shared settings object in a process	Restrict a class to one instance in a scope	Global state, testing, and scope surprises

Simple Factory is a common idiom, distinct from the named Factory Method pattern. Abstract Factory creates a related family; it need not literally be a factory that returns factories.

A normal singleton does not create one object across every server. A dependency injection container can manage a shared object in its own scope without proving uniqueness across a cluster.

Connecting objects

Pattern	Simple picture	Software use	Watch for
Adapter	Convert an old paper form into the new format	Wrap a legacy catalog interface	Map errors and meanings, not just names
Bridge	Lesson type and display medium vary separately	Separate abstraction from implementation	Unnecessary abstractions for one fixed case
Composite	Count pages in one file or a folder of files	Treat leaves and groups uniformly	Make valid operations clear for both
Decorator	Add a protective cover around a notebook	Wrap behavior with extra features	Wrapper order can change behavior
Facade	One office request coordinates several departments	Offer a simple interface to a subsystem	Avoid a facade that owns every business rule
Flyweight	Reuse one font definition for many letters	Share intrinsic immutable data	Keep per-item state outside shared objects
Proxy	A desk controls access to a stored document	Access checks, lazy loading, remote access	Do not hide important latency or failure

An Adapter changes an interface. A Decorator adds behavior while preserving the useful contract. A Proxy controls access. A Facade simplifies a subsystem. Their shapes may look alike, so explain their intent.

Organizing behavior

Pattern	Simple picture	Software use	Watch for
Chain of Responsibility	A request passes through suitable checkers	Validation or request handlers	Define stop conditions and unhandled cases
Command	A work request written on a card	Queue, record, or retry an action	Undo may be impossible or require compensation
Iterator	Turn through pages without knowing the binding	Traverse a collection through one interface	Mutation during traversal
Mediator	An event coordinator manages participants	Centralize interactions among objects	The coordinator can become too large
Memento	Save a drawing before a large edit	Capture state for later restoration	Snapshot size and sensitive data
Observer	Class display refreshes when its model changes	Notify dependent objects of a change	Slow observers and subscription lifetime
Visitor	Different inspectors examine the same document tree	Add operations across stable element types	New element types affect visitors
Strategy	Choose walking or bus directions	Swap an algorithm behind one contract	Excess abstractions for trivial choices
State	A reservation behaves differently after collection	Let state determine allowed behavior	Guard invalid transitions
Template Method	A fixed lesson routine with custom activities	Subclasses fill steps in an algorithm	Rigid inheritance and surprising hooks

An in-process Observer usually does not offer the durable delivery of a broker. Command objects do not automatically create database transactions. Mementos capture state; Commands capture requested behavior.

Strategy versus State

Use Strategy when selecting one of several ways to do the same job. A report may be exported as CSV or plain text.

Use State when behavior follows the lifecycle. A collected reservation cannot be collected again. This state diagram shows domain rules; a State-pattern implementation is one way to encode them.

stateDiagram-v2
    [*] --> Reserved
    Reserved --> Ready: Copy prepared
    Reserved --> Cancelled: Student cancels
    Ready --> Collected: Student collects
    Ready --> Expired: Pickup deadline passes
    Collected --> [*]
    Cancelled --> [*]
    Expired --> [*]

A tiny original Strategy example

This Python example demonstrates structure, not production export handling.

from typing import Protocol

class TitleFormatter(Protocol):
    def format(self, titles: list[str]) -> str: ...

class LineFormatter:
    def format(self, titles: list[str]) -> str:
        return "\n".join(titles)

class NumberedFormatter:
    def format(self, titles: list[str]) -> str:
        return "\n".join(
            f"{number}. {title}"
            for number, title in enumerate(titles, start=1)
        )

class ReadingListReport:
    def __init__(self, formatter: TitleFormatter):
        self.formatter = formatter

    def render(self, titles: list[str]) -> str:
        return self.formatter.format(titles)

report = ReadingListReport(NumberedFormatter())
print(report.render(["Clouds", "Rivers"]))

The report delegates formatting through a contract. Another formatter can be passed in without adding a formatting switch inside the report. For a tiny program, a plain function can be sufficient; classes are not mandatory.

SOLID in plain language

Principle	Practical meaning
Single Responsibility	Group code around one reason to change, not literally one method
Open/Closed	Allow expected extensions without editing stable core logic everywhere
Liskov Substitution	A replacement must honor the behavior clients rely on
Interface Segregation	Clients should not depend on operations they do not need
Dependency Inversion	High-level policy depends on useful abstractions, not concrete plumbing

Use these principles to discuss trade-offs. A new interface is helpful when it protects a real boundary; adding one for every class can make the code harder to follow.

12. Interview practice: questions 21–30

Q21. Can a monolith scale horizontally?

Yes. Multiple instances can serve requests behind a balancer. The design must handle state, shared data, and downstream capacity. Independent scaling of one internal capability is harder than scaling the whole app.

Q22. When should work be asynchronous?

When the user can accept a later result and the system can durably track that work. I would expose acceptance, progress, failure, and completion rather than return success too early.

Q23. How do queues help with a traffic burst?

They buffer work and separate producer and consumer rates. They cannot absorb unlimited overload, so I would monitor age and backlog and apply backpressure.

Q24. What happens if a worker crashes after its database commit?

If acknowledgment was not recorded, the message can return. A processed-event marker committed with the database effect makes that retry safe. External effects need their own protection.

Q25. What problem does an outbox solve?

It atomically stores a business change and the obligation to publish its event in one local transaction. A relay completes publication later, with duplicate handling at consumers.

Q26. Does a saga roll back the whole distributed system?

No. It uses local transactions and compensating business actions. Intermediate states are visible, compensation may fail, and reconciliation may be required.

Q27. Are CQRS and event sourcing the same?

No. CQRS separates write and read models. Event sourcing makes event history the source of truth. Either can be used without the other.

Q28. Strategy or State?

Strategy selects an interchangeable way to do work. State changes permitted behavior as an object's lifecycle changes. I explain the intent before the class structure.

Q29. Adapter or Facade for an old catalog system?

Use Adapter to match the interface expected by the new application. Use Facade to simplify a collection of operations. Both may be useful, but they solve different problems.

Q30. Why avoid using patterns everywhere?

Each adds concepts and indirection. I use a pattern when it makes an actual change, boundary, or behavior easier to manage, and keep direct code when it is clearer.

Quick revision

Separate responsibilities before splitting deployments. A queue needs capacity limits and ownership. Design for duplicate delivery and uncertain outcomes. Keep outbox, saga, CQRS, and event sourcing distinct. Choose object patterns by the problem they solve.