The first time you join a microservices project, the code is rarely what overwhelms you. It’s a meeting like this: “Order service publishes the event through the outbox, the payment consumer has to be idempotent, if it fails the saga compensates, and don’t forget the correlation ID for tracing.” Every word is English, and you still have no idea what was said.

There are plenty of glossaries. The problem is that isolated definitions don’t stick, because you never see what problem each term was invented to fix.

This post goes the other way: we follow one order request, from the moment a user taps “Place order” until the order is confirmed. Every time something can go wrong, the matching term shows up. This is the first post in the Microservices A–Z series. Each group of terms below gets its own deep-dive post later, with .NET code.

The example system#

A small online shop, split into 4 services:

• Order Service: creates and manages orders
• Payment Service: charges the customer
• Inventory Service: reserves stock
• Notification Service: sends email/SMS

flowchart LR
    C["Client (Web/Mobile)"] --> GW["API Gateway"]
    GW --> O["Order Service"]
    O --> ODB[("Order DB")]
    O -- "OrderCreated" --> MB{{"Message Broker"}}
    MB --> P["Payment Service"]
    MB --> I["Inventory Service"]
    MB --> N["Notification Service"]
    P --> PDB[("Payment DB")]
    I --> IDB[("Inventory DB")]

The whole series reuses exactly this system.

Stage 0: Why split at all?#

Before there were 4 services, the system usually started as one block.

WARNING

Microservices solve an organizational and team-scaling problem before they solve a technical one. If your team has 3 people, you almost certainly don’t need them yet. P2 covers this in detail.

Stage 1: Splitting it right — DDD#

First question: where is the boundary between Order and Payment? Get it wrong and the two services end up calling each other constantly. That’s a distributed monolith: worse than the monolith you started with.

Stage 2: The request comes in — Communication & APIs#

The user taps “Place order”. The request leaves the mobile app and enters the system.

Order Service receives the request and saves the order as Pending. Now it has to tell Payment and Inventory.

Stage 3: Telling other services — Messaging#

Order could call Payment directly over HTTP. But if Payment is down, the order goes down with it. The more common approach is to fire an event and move on.

Stage 4: Data is scattered — Consistency#

This is the hardest stage, and the one system design interviews dig into the most.

Each service keeps its own database (Database per Service). So “create order + charge + reserve stock” no longer fits in one transaction. What happens if the charge succeeds and then Inventory says the item is out of stock?

Outbox with EF Core, the core idea is just this:

await using var tx = await db.Database.BeginTransactionAsync(ct);

db.Orders.Add(order);
db.OutboxMessages.Add(OutboxMessage.From(new OrderCreated(order.Id, order.Total)));

await db.SaveChangesAsync(ct);   // order and message commit together
await tx.CommitAsync(ct);
// Background worker reads unsent OutboxMessages → publishes to the broker → marks them sent

And on the consumer side, idempotency by remembering which messages were already handled:

public async Task Consume(ConsumeContext<OrderCreated> context)
{
    var messageId = context.MessageId!.Value;
    if (await db.ProcessedMessages.AnyAsync(m => m.Id == messageId))
        return; // already handled, skip

    await paymentService.ChargeAsync(context.Message.OrderId, context.Message.Total);
    db.ProcessedMessages.Add(new ProcessedMessage(messageId));
    await db.SaveChangesAsync();
}

TIP

MassTransit and Wolverine both ship Outbox/Inbox support. In a real project, use it instead of rolling your own. The snippets above are only there to show the mechanism. The consumer also has a hole: if two copies of the same message arrive concurrently, both pass the AnyAsync check. The real version needs a unique constraint on ProcessedMessages.Id, written in the same transaction as the business change. The Idempotent Consumer post goes into detail.

Stage 5: A service gets slow — Resilience#

Payment Service calls an external payment gateway that sometimes takes 30 seconds to respond. Do nothing, and Payment’s threads all get stuck, then Payment dies, then everything calling Payment dies too. That’s a cascading failure.

Since .NET 8, the Microsoft.Extensions.Http.Resilience package bundles these for HttpClient:

builder.Services.AddHttpClient<PaymentGatewayClient>()
    .AddStandardResilienceHandler(); // timeout + retry + circuit breaker, built on Polly v8

Stage 6: A bug in production — Observability#

A customer reports: “I was charged but my order is still Pending.” That request went through 4 services and a broker. Logs live in 4 different places. How do you find it?

NOTE

Observability isn’t “we have logs.” The real question is: when an incident you’ve never seen before happens, can you answer “what is going on” using only the data the system already emits?

Stage 7: Deploying and operating#

Order Service has a new version. How do you deploy it without disrupting the other 3 services?

I’ve written about Feature Flags in .NET in a separate post if you want to read ahead.

The order’s whole journey#

sequenceDiagram
    participant C as Client
    participant GW as API Gateway
    participant O as Order Service
    participant MB as Broker
    participant P as Payment
    participant I as Inventory

    C->>GW: POST /orders (JWT)
    GW->>O: route + Correlation ID
    O->>O: Save Order + Outbox (1 transaction)
    O-->>C: 202 Accepted (Pending)
    O->>MB: OrderCreated (outbox worker)
    MB->>P: OrderCreated
    P->>P: Idempotent check → charge (retry, circuit breaker)
    P->>MB: PaymentSucceeded
    MB->>I: PaymentSucceeded
    I-->>MB: InventoryFailed (out of stock)
    MB->>P: InventoryFailed → Refund (compensate)
    MB->>O: InventoryFailed → Order Cancelled

Reread the meeting from the start: “Order service publishes the event through the outbox, the payment consumer has to be idempotent, if it fails the saga compensates, and don’t forget the correlation ID for tracing.” Every phrase now has a place in the diagram above.

Next in the series#

The series has 22 posts, following the same stages:

1. Foundations: this post, and P2: When not to use microservices
2. Decomposition: DDD & Bounded Context, Database per Service, Strangler Fig & ACL
3. Communication: API Gateway & BFF with YARP, Messaging, Kafka vs RabbitMQ
4. Data consistency: Saga, Outbox, Idempotent Consumer, CQRS, Event Sourcing
5. Resilience: Retry, Timeout, Circuit Breaker, Bulkhead with Polly
6. Observability: Logging, Tracing, Health checks with OpenTelemetry
7. Deployment: Service Discovery, Service Mesh, Canary, Contract Testing

NOTE

References for this series: microservices.vn (a thorough Vietnamese catalog of terms and patterns), Chris Richardson’s microservices.io, and Building Microservices (Sam Newman, 2nd edition).