LLD Series: Payment Gateway with Routing, Retries and Webhooks (Low-Level Design)
Overview
- 1. Problem statement and how it is asked
- 2. Requirements clarification
- 3. Entities, relationships, class diagram
- 4. Design patterns and why
- 5. Complete Java code
- 6. Walkthrough of key flows
- 7. Extensibility
- 8. Concurrency and edge cases
- 9. Follow-up questions and model answers
- 10. Common mistakes, red flags, rubric
Part of the Interview Prep series. Dotted-underline words are glossary terms: hover for the meaning or open the Glossary. Track your progress on the Interview Prep tracker. Series index: Start here.
1. Problem statement and how it is asked
Typical phrasing: "Design a payment gateway (Razorpay / PayU / Stripe style). Merchants submit payments by card, UPI, net-banking or wallet. The gateway routes to a bank/acquirer, falls back if one is down, retries safely, never double-charges, tracks a payment state machine and notifies merchants through webhooks."
- Asked at: PayU, Visa, Mastercard, NPCI, Amex - this is the "home turf" question for payments companies; expect it as the senior/hard round or as an extended follow-up to wallet.
- Interviewer is checking: Strategy/Factory for payment methods, routing + fallback that knows which failures are safe to retry, idempotency, an explicit state machine, async results (UPI callbacks), webhooks (signed, retried), and money-safety reasoning.
2. Requirements clarification
Functional
pay(request): merchant, order id, amount, method (CARD / UPI / NET_BANKING / WALLET), method details, idempotency key.- Validate method-specific details (Luhn for cards, VPA format, bank code).
- Route to a capable connector (acquirer/bank adapter). Support static priority and success-rate based routing.
- Retry transient errors on the same connector with exponential backoff, then fail over to the next connector. Never retry or fail over on a hard decline.
- Circuit breaker per connector to stop hammering a failing bank.
- Payment lifecycle: CREATED -> PROCESSING -> (SUCCESS | FAILED | PENDING) ; PENDING -> (SUCCESS | FAILED) ; SUCCESS -> REFUNDED.
- Async confirmation endpoint (bank callback) that is idempotent.
- Webhooks to merchants on SUCCESS/FAILED/REFUNDED: HMAC-signed, retried with backoff, dead-lettered when exhausted.
- Refund of a successful payment (once).
Non-functional: no double charge (idempotency + state machine); low latency (routing in-memory); extensibility (new method/bank/routing rule without touching the orchestrator); observability (attempt trail per payment); PCI hygiene (raw PAN never stored/logged, masked in Instrument).
Assumptions: connectors are synchronous adapters (a real one wraps HTTP/ISO 8583 with timeouts); a TECHNICAL_ERROR means the bank definitely did not process the request (e.g. connection refused/5xx before processing). Ambiguous timeouts are treated in follow-ups (status inquiry before retry). Amounts in minor units.
Out of scope: settlement/reconciliation files, 3-D Secure/OTP step, tokenisation vault, fraud engine, multi-currency/FX, chargebacks, persistence (discussed).
3. Entities, relationships, class diagram
PaymentRequest ---> PaymentService (orchestrator) ---creates---> Payment (state machine)
| | | | | | status, attempts[], bankRef
| | | | +--publishes--> PaymentEvent
| | | | |
+------------------+ | | +----------+ v
v v v v <<iface>> PaymentListener
PaymentMethodFactory RoutingStrategy RetryPolicy |-- WebhookDispatcher (HMAC, retries)
| creates/gets ^ ^ + Sleeper |-- AuditListener (not shown)
v | |
<<iface>> PaymentMethod StaticPriority SuccessRate HealthRegistry --> ConnectorHealth (circuit breaker + success rate)
^ ^ ^ ^
Card Upi NetBanking Wallet <<iface>> BankConnector (adapter per acquirer/bank)
| prepare(details) -> Instrument ^ ^ ^
| (validate, mask) HDFC adapter ICICI adapter NPCI-UPI adapter (here: ScriptedConnector fakes)
Payment state machine:
CREATED --> PROCESSING --+--> SUCCESS --> REFUNDED
| ^
+--> PENDING --+--> FAILED
| (bank callback)
+--> FAILED
FAILED and REFUNDED are terminal. Encoded in PaymentStatus.next(); Payment.transition rejects anything else.
4. Design patterns and why
| Pattern | Where | Why / alternative rejected |
|---|---|---|
| Strategy | PaymentMethod (validation + sanitising per method), RoutingStrategy, RetryPolicy | Behaviour varies independently. Rejected: switch(methodType) in the orchestrator; every new method edits the hot path. |
| Factory (registry) | PaymentMethodFactory | One place maps type -> strategy; register() supports plug-ins. |
| Adapter | BankConnector | Each bank has its own protocol; the orchestrator sees one interface. |
| State machine | PaymentStatus + Payment | Money-critical lifecycle; illegal transitions throw; callbacks use compare-and-transition so duplicates are no-ops. Rejected: free-form string statuses. |
| Observer | PaymentListener, WebhookDispatcher | Merchant notifications, audit, analytics are side effects that must not be in the payment path or break it. |
| Circuit breaker | ConnectorHealth | Fail fast to the next bank instead of burning latency on a dead one; half-open after cooldown. |
| Idempotent receiver | byIdempotencyKey map | Safe client retries (network timeouts are the norm). |
| (Mention) Chain of Responsibility | fallback chain over connectors | The ordered fallback loop is a CoR; kept as a loop for readability. |
| (Mention) Template Method / Decorator | RetryingConnector wrapper | Alternative: decorate each connector with retry/timeout/metrics; we keep retry in the orchestrator because failover needs cross-connector knowledge. |
5. Complete Java code
Default package; javac *.java && java Main. Fake bank connectors (ScriptedConnector) play scripted outcomes so every path is deterministic.
5.1 Enums, state machine, exceptions
// Enums.java
enum PaymentMethodType { CARD, UPI, NET_BANKING, WALLET }
/** What a bank/acquirer answered. Only TECHNICAL_ERROR is safe to retry / fail over. */
enum AuthOutcome { APPROVED, DECLINED, TECHNICAL_ERROR, PENDING }
// PaymentStatus.java
import java.util.EnumSet;
import java.util.Set;
/** The payment state machine. Illegal transitions are impossible by construction. */
public enum PaymentStatus {
CREATED, PROCESSING, PENDING, SUCCESS, FAILED, REFUNDED;
public Set<PaymentStatus> next() {
switch (this) {
case CREATED: return EnumSet.of(PROCESSING);
case PROCESSING: return EnumSet.of(SUCCESS, FAILED, PENDING);
case PENDING: return EnumSet.of(SUCCESS, FAILED);
case SUCCESS: return EnumSet.of(REFUNDED);
default: return EnumSet.noneOf(PaymentStatus.class); // FAILED, REFUNDED are terminal
}
}
public boolean canMoveTo(PaymentStatus to) { return next().contains(to); }
}
// GatewayException.java
public class GatewayException extends RuntimeException {
public GatewayException(String m) { super(m); }
}
// PaymentValidationException.java
public class PaymentValidationException extends GatewayException {
public PaymentValidationException(String m) { super(m); }
}
5.2 Request, instrument and payment methods (Strategy + Factory)
// PaymentRequest.java
import java.util.Map;
public final class PaymentRequest {
public final String merchantId, orderId, idempotencyKey;
public final long amountMinor;
public final PaymentMethodType methodType;
public final Map<String, String> details; // card number, vpa, bank code ...
public PaymentRequest(String merchantId, String orderId, String idempotencyKey, long amountMinor,
PaymentMethodType methodType, Map<String, String> details) {
if (amountMinor <= 0) throw new PaymentValidationException("Amount must be positive");
if (idempotencyKey == null || idempotencyKey.isBlank()) throw new PaymentValidationException("Idempotency key required");
this.merchantId = merchantId; this.orderId = orderId; this.idempotencyKey = idempotencyKey;
this.amountMinor = amountMinor; this.methodType = methodType; this.details = Map.copyOf(details);
}
/** Same key + different fingerprint => client bug (409 Conflict). Sensitive details excluded from the hash input shown in logs. */
String fingerprint() { return merchantId + "|" + orderId + "|" + amountMinor + "|" + methodType + "|" + details.hashCode(); }
}
// Instrument.java
import java.util.Map;
/** Validated, sanitised view of how the customer wants to pay. Never carries the raw PAN/CVV past the method class. */
public final class Instrument {
public final PaymentMethodType type; public final String maskedRef; public final Map<String, String> attributes;
public Instrument(PaymentMethodType type, String maskedRef, Map<String, String> attributes) {
this.type = type; this.maskedRef = maskedRef; this.attributes = Map.copyOf(attributes);
}
public String attr(String k) { return attributes.getOrDefault(k, ""); }
}
// PaymentMethod.java
import java.util.Map;
/** Strategy: validates method-specific details and produces a sanitised Instrument. */
public interface PaymentMethod {
PaymentMethodType type();
Instrument prepare(Map<String, String> details); // throws PaymentValidationException
}
// CardPaymentMethod.java
import java.util.Map;
public final class CardPaymentMethod implements PaymentMethod {
@Override public PaymentMethodType type() { return PaymentMethodType.CARD; }
@Override public Instrument prepare(Map<String, String> d) {
String pan = d.getOrDefault("pan", "").replaceAll("\\s", "");
if (!pan.matches("\\d{13,19}") || !luhn(pan)) throw new PaymentValidationException("Invalid card number");
if (!d.getOrDefault("cvv", "").matches("\\d{3,4}")) throw new PaymentValidationException("Invalid CVV");
if (!d.getOrDefault("expiry", "").matches("(0[1-9]|1[0-2])/\\d{2}")) throw new PaymentValidationException("Invalid expiry");
String network = pan.startsWith("4") ? "VISA" : pan.startsWith("5") ? "MASTERCARD" : "OTHER";
return new Instrument(type(), "****" + pan.substring(pan.length() - 4), Map.of("network", network));
}
static boolean luhn(String s) {
int sum = 0; boolean dbl = false;
for (int i = s.length() - 1; i >= 0; i--) {
int n = s.charAt(i) - '0';
if (dbl && (n *= 2) > 9) n -= 9;
sum += n; dbl = !dbl;
}
return sum % 10 == 0;
}
}
// UpiPaymentMethod.java
import java.util.Map;
public final class UpiPaymentMethod implements PaymentMethod {
@Override public PaymentMethodType type() { return PaymentMethodType.UPI; }
@Override public Instrument prepare(Map<String, String> d) {
String vpa = d.getOrDefault("vpa", "");
if (!vpa.matches("[\\w.\\-]{2,}@[a-zA-Z]{2,}")) throw new PaymentValidationException("Invalid VPA");
return new Instrument(type(), vpa, Map.of("handle", vpa.substring(vpa.indexOf('@') + 1)));
}
}
// NetBankingPaymentMethod.java
import java.util.Map;
import java.util.Set;
public final class NetBankingPaymentMethod implements PaymentMethod {
private static final Set<String> BANKS = Set.of("HDFC", "ICICI", "SBI", "AXIS");
@Override public PaymentMethodType type() { return PaymentMethodType.NET_BANKING; }
@Override public Instrument prepare(Map<String, String> d) {
String bank = d.getOrDefault("bank", "");
if (!BANKS.contains(bank)) throw new PaymentValidationException("Unsupported bank " + bank);
return new Instrument(type(), bank, Map.of("bank", bank));
}
}
// WalletPaymentMethod.java
import java.util.Map;
public final class WalletPaymentMethod implements PaymentMethod {
@Override public PaymentMethodType type() { return PaymentMethodType.WALLET; }
@Override public Instrument prepare(Map<String, String> d) {
String id = d.getOrDefault("walletId", "");
if (id.isBlank()) throw new PaymentValidationException("walletId required");
return new Instrument(type(), id, Map.of());
}
}
// PaymentMethodFactory.java
import java.util.EnumMap;
import java.util.Map;
/** Registry-style factory: new method = new class + register(); service untouched. */
public final class PaymentMethodFactory {
private final Map<PaymentMethodType, PaymentMethod> methods = new EnumMap<>(PaymentMethodType.class);
public static PaymentMethodFactory withDefaults() {
PaymentMethodFactory f = new PaymentMethodFactory();
f.register(new CardPaymentMethod()); f.register(new UpiPaymentMethod());
f.register(new NetBankingPaymentMethod()); f.register(new WalletPaymentMethod());
return f;
}
public void register(PaymentMethod m) { methods.put(m.type(), m); }
public PaymentMethod get(PaymentMethodType t) {
PaymentMethod m = methods.get(t);
if (m == null) throw new PaymentValidationException("Unsupported payment method " + t);
return m;
}
}
5.3 Bank connectors (Adapter) and health (circuit breaker)
// AuthRequest.java
public final class AuthRequest {
public final String paymentId; public final long amountMinor; public final Instrument instrument;
public AuthRequest(String paymentId, long amountMinor, Instrument instrument) {
this.paymentId = paymentId; this.amountMinor = amountMinor; this.instrument = instrument;
}
}
// AuthResponse.java
public final class AuthResponse {
public final AuthOutcome outcome; public final String code, bankRef;
private AuthResponse(AuthOutcome o, String code, String ref) { this.outcome = o; this.code = code; this.bankRef = ref; }
public static AuthResponse approved(String ref) { return new AuthResponse(AuthOutcome.APPROVED, "00", ref); }
public static AuthResponse declined(String code) { return new AuthResponse(AuthOutcome.DECLINED, code, null); }
public static AuthResponse technical(String code) { return new AuthResponse(AuthOutcome.TECHNICAL_ERROR, code, null); }
public static AuthResponse pending(String ref) { return new AuthResponse(AuthOutcome.PENDING, "PENDING", ref); }
}
// BankConnector.java
/** Adapter to one acquirer / bank / network. One implementation per integration (HDFC, ICICI, NPCI-UPI ...). */
public interface BankConnector {
String name();
boolean supports(PaymentMethodType type);
AuthResponse authorize(AuthRequest request);
boolean refund(String bankRef, long amountMinor);
}
// ScriptedConnector.java
import java.util.Set;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Supplier;
/** Test double: plays back a script of responses, then falls back to a default behaviour. */
public final class ScriptedConnector implements BankConnector {
private final String name; private final Set<PaymentMethodType> supported;
private final ConcurrentLinkedQueue<AuthResponse> script = new ConcurrentLinkedQueue<>();
private final Supplier<AuthResponse> fallback;
private final AtomicInteger calls = new AtomicInteger();
public ScriptedConnector(String name, Set<PaymentMethodType> supported, Supplier<AuthResponse> fallback) {
this.name = name; this.supported = supported; this.fallback = fallback;
}
public ScriptedConnector thenReturn(AuthResponse... rs) { for (AuthResponse r : rs) script.add(r); return this; }
public int calls() { return calls.get(); }
@Override public String name() { return name; }
@Override public boolean supports(PaymentMethodType t) { return supported.contains(t); }
@Override public AuthResponse authorize(AuthRequest r) {
int n = calls.incrementAndGet();
AuthResponse scripted = script.poll();
return scripted != null ? scripted : fallback.get();
}
@Override public boolean refund(String ref, long amt) { return true; }
}
// ConnectorHealth.java
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
/** Circuit breaker + success-rate counters for one connector. Lock-free (atomics). */
public final class ConnectorHealth {
private final int failureThreshold; private final Duration cooldown;
private final AtomicInteger consecutiveFailures = new AtomicInteger();
private final AtomicLong ok = new AtomicLong(), total = new AtomicLong();
private volatile Instant openUntil = Instant.MIN;
public ConnectorHealth(int failureThreshold, Duration cooldown) { this.failureThreshold = failureThreshold; this.cooldown = cooldown; }
/** CLOSED or half-open (cooldown elapsed) => allow a trial request. */
public boolean allowRequest(Clock clock) { return !Instant.now(clock).isBefore(openUntil); }
public void recordSuccess() { consecutiveFailures.set(0); ok.incrementAndGet(); total.incrementAndGet(); }
public void recordFailure(Clock clock) {
total.incrementAndGet();
if (consecutiveFailures.incrementAndGet() >= failureThreshold) openUntil = Instant.now(clock).plus(cooldown);
}
public double successRate() { long t = total.get(); return t == 0 ? 1.0 : (double) ok.get() / t; }
public boolean isOpen(Clock clock) { return !allowRequest(clock); }
}
// HealthRegistry.java
import java.time.Duration;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
public final class HealthRegistry {
private final Map<String, ConnectorHealth> byConnector = new ConcurrentHashMap<>();
private final int threshold; private final Duration cooldown;
public HealthRegistry(int threshold, Duration cooldown) { this.threshold = threshold; this.cooldown = cooldown; }
public ConnectorHealth of(String connectorName) {
return byConnector.computeIfAbsent(connectorName, n -> new ConnectorHealth(threshold, cooldown));
}
}
5.4 Routing and retry
// RoutingStrategy.java
import java.util.List;
/** Strategy: order the capable connectors for this payment. First = preferred, rest = fallbacks. */
public interface RoutingStrategy {
List<BankConnector> route(PaymentRequest request, Instrument instrument, List<BankConnector> capable, HealthRegistry health);
}
// StaticPriorityRouting.java
import java.util.List;
/** Keep configured order (e.g. cheapest MDR first). */
public final class StaticPriorityRouting implements RoutingStrategy {
@Override public List<BankConnector> route(PaymentRequest r, Instrument i, List<BankConnector> capable, HealthRegistry h) {
return capable;
}
}
// SuccessRateRouting.java
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
/** Smart routing: best live success-rate first (stable sort keeps configured order on ties). */
public final class SuccessRateRouting implements RoutingStrategy {
@Override public List<BankConnector> route(PaymentRequest r, Instrument i, List<BankConnector> capable, HealthRegistry h) {
List<BankConnector> sorted = new ArrayList<>(capable);
sorted.sort(Comparator.comparingDouble((BankConnector c) -> h.of(c.name()).successRate()).reversed());
return sorted;
}
}
// RetryPolicy.java
public final class RetryPolicy {
private final int maxAttemptsPerConnector; private final long baseDelayMs;
public RetryPolicy(int maxAttemptsPerConnector, long baseDelayMs) { this.maxAttemptsPerConnector = maxAttemptsPerConnector; this.baseDelayMs = baseDelayMs; }
public int maxAttempts() { return maxAttemptsPerConnector; }
/** Exponential backoff before attempt n (n >= 2): base * 2^(n-2). Production adds jitter. */
public long delayBefore(int attempt) { return baseDelayMs << (attempt - 2); }
}
// Sleeper.java
/** Injectable so tests and the demo do not really sleep. */
public interface Sleeper { void sleep(long millis); }
5.5 Payment aggregate, events, webhooks (Observer)
// Payment.java
import java.util.ArrayList;
import java.util.List;
/** Aggregate with a guarded state machine. All mutation goes through synchronized methods of this object (per-payment lock). */
public final class Payment {
private final String id, fingerprint; private final PaymentRequest request; private final Instrument instrument;
private PaymentStatus status = PaymentStatus.CREATED;
private final List<String> attempts = new ArrayList<>();
private String connectorName, bankRef, failureReason;
Payment(String id, PaymentRequest request, Instrument instrument) {
this.id = id; this.request = request; this.instrument = instrument; this.fingerprint = request.fingerprint();
}
public String id() { return id; }
public PaymentRequest request() { return request; }
public Instrument instrument() { return instrument; }
String fingerprint() { return fingerprint; }
public synchronized PaymentStatus status() { return status; }
public synchronized String bankRef() { return bankRef; }
public synchronized String connectorName() { return connectorName; }
public synchronized String failureReason() { return failureReason; }
public synchronized List<String> attempts() { return List.copyOf(attempts); }
synchronized void transition(PaymentStatus to) {
if (!status.canMoveTo(to)) throw new GatewayException("Illegal transition " + status + " -> " + to + " for " + id);
status = to;
}
/** Atomic compare-and-transition: used by webhooks/callbacks that may be duplicated or race. */
synchronized boolean transitionIf(PaymentStatus expected, PaymentStatus to) {
if (status != expected) return false;
transition(to);
return true;
}
synchronized void addAttempt(String connector, String result) { attempts.add(connector + ":" + result); }
synchronized void recordRoute(String connector, String ref) { this.connectorName = connector; this.bankRef = ref; }
synchronized void recordFailure(String reason) { this.failureReason = reason; }
@Override public synchronized String toString() {
return id + " " + request.orderId + " " + status + (failureReason != null ? " (" + failureReason + ")" : "")
+ " via " + connectorName + " trail=" + attempts;
}
}
// PaymentEvent.java
public final class PaymentEvent {
public final String paymentId, merchantId, orderId, bankRef, reason; public final PaymentStatus status; public final long amountMinor;
public PaymentEvent(Payment p) {
this.paymentId = p.id(); this.merchantId = p.request().merchantId; this.orderId = p.request().orderId;
this.status = p.status(); this.amountMinor = p.request().amountMinor; this.bankRef = p.bankRef(); this.reason = p.failureReason();
}
}
// PaymentListener.java
/** Observer. Called synchronously after every state change, outside any payment lock. Must not throw. */
public interface PaymentListener { void onEvent(PaymentEvent event); }
// WebhookClient.java
/** HTTP port to a merchant endpoint. Returns true on 2xx. */
public interface WebhookClient { boolean post(String url, String jsonBody, String signature); }
// WebhookDispatcher.java
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
/** Observer that notifies merchants of terminal outcomes: signed payload, retried with backoff, dead-lettered when exhausted. */
public final class WebhookDispatcher implements PaymentListener {
public static final class Endpoint {
final String url, secret; final WebhookClient client;
public Endpoint(String url, String secret, WebhookClient client) { this.url = url; this.secret = secret; this.client = client; }
}
private static final Set<PaymentStatus> NOTIFY_ON = Set.of(PaymentStatus.SUCCESS, PaymentStatus.FAILED, PaymentStatus.REFUNDED);
private final Map<String, Endpoint> endpoints = new ConcurrentHashMap<>();
private final Sleeper sleeper; private final int maxAttempts;
public WebhookDispatcher(Sleeper sleeper, int maxAttempts) { this.sleeper = sleeper; this.maxAttempts = maxAttempts; }
public void register(String merchantId, Endpoint e) { endpoints.put(merchantId, e); }
@Override public void onEvent(PaymentEvent e) {
Endpoint ep = endpoints.get(e.merchantId);
if (ep == null || !NOTIFY_ON.contains(e.status)) return;
String body = "{\"paymentId\":\"" + e.paymentId + "\",\"orderId\":\"" + e.orderId + "\",\"status\":\"" + e.status
+ "\",\"amount\":" + e.amountMinor + "}";
String sig = hmacSha256Hex(ep.secret, body);
for (int attempt = 1; attempt <= maxAttempts; attempt++) {
if (attempt > 1) sleeper.sleep(100L << (attempt - 2));
boolean ok = false;
try { ok = ep.client.post(ep.url, body, sig); } catch (RuntimeException ignored) { }
System.out.println(" [webhook] " + e.paymentId + " " + e.status + " attempt " + attempt + (ok ? " delivered" : " failed"));
if (ok) return;
}
System.out.println(" [webhook] " + e.paymentId + " moved to dead-letter queue for manual replay");
}
static String hmacSha256Hex(String secret, String data) {
try {
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(secret.getBytes(StandardCharsets.UTF_8), "HmacSHA256"));
StringBuilder sb = new StringBuilder();
for (byte b : mac.doFinal(data.getBytes(StandardCharsets.UTF_8))) sb.append(String.format("%02x", b));
return sb.toString();
} catch (Exception ex) { throw new IllegalStateException(ex); }
}
}
5.6 Orchestrator
// PaymentService.java
import java.time.Clock;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.atomic.AtomicLong;
import java.util.stream.Collectors;
/**
* Orchestrator. Thread-safety:
* - idempotency: ConcurrentHashMap.putIfAbsent => exactly one thread processes a (merchant,key); others get the same Payment.
* - state changes: per-Payment monitor (Payment.transition*), so unrelated payments never contend.
* - listeners/webhooks are invoked after the transition, never while holding a payment lock.
*/
public final class PaymentService {
private final PaymentMethodFactory methods;
private final List<BankConnector> connectors;
private final RoutingStrategy routing;
private final RetryPolicy retry;
private final HealthRegistry health;
private final Sleeper sleeper;
private final Clock clock;
private final Map<String, Payment> byIdempotencyKey = new ConcurrentHashMap<>();
private final Map<String, Payment> byId = new ConcurrentHashMap<>();
private final List<PaymentListener> listeners = new CopyOnWriteArrayList<>();
private final AtomicLong seq = new AtomicLong();
public PaymentService(PaymentMethodFactory methods, List<BankConnector> connectors, RoutingStrategy routing,
RetryPolicy retry, HealthRegistry health, Sleeper sleeper, Clock clock) {
this.methods = methods; this.connectors = List.copyOf(connectors); this.routing = routing;
this.retry = retry; this.health = health; this.sleeper = sleeper; this.clock = clock;
}
public void addListener(PaymentListener l) { listeners.add(l); }
public Payment pay(PaymentRequest req) {
Instrument instrument = methods.get(req.methodType).prepare(req.details); // validation first, no side effects
Payment fresh = new Payment("PAY-" + seq.incrementAndGet(), req, instrument);
String key = req.merchantId + ":" + req.idempotencyKey;
Payment existing = byIdempotencyKey.putIfAbsent(key, fresh);
if (existing != null) {
if (!existing.fingerprint().equals(fresh.fingerprint()))
throw new GatewayException("Idempotency key reused with a different request: " + req.idempotencyKey);
return existing; // replay: same resource, no new bank call
}
byId.put(fresh.id(), fresh);
process(fresh);
return fresh;
}
private void process(Payment p) {
p.transition(PaymentStatus.PROCESSING);
publish(p);
List<BankConnector> capable = connectors.stream().filter(c -> c.supports(p.request().methodType)).collect(Collectors.toList());
List<BankConnector> ordered = routing.route(p.request(), p.instrument(), capable, health);
AuthRequest auth = new AuthRequest(p.id(), p.request().amountMinor, p.instrument());
for (BankConnector connector : ordered) {
ConnectorHealth h = health.of(connector.name());
for (int attempt = 1; attempt <= retry.maxAttempts(); attempt++) {
if (!h.allowRequest(clock)) { p.addAttempt(connector.name(), "SKIPPED_CIRCUIT_OPEN"); break; }
if (attempt > 1) sleeper.sleep(retry.delayBefore(attempt));
AuthResponse resp = connector.authorize(auth);
p.addAttempt(connector.name(), resp.outcome.name());
switch (resp.outcome) {
case APPROVED:
h.recordSuccess(); p.recordRoute(connector.name(), resp.bankRef);
p.transition(PaymentStatus.SUCCESS); publish(p); return;
case PENDING: // e.g. UPI collect: final status arrives via bank callback
h.recordSuccess(); p.recordRoute(connector.name(), resp.bankRef);
p.transition(PaymentStatus.PENDING); publish(p); return;
case DECLINED: // business decision: do NOT retry or fail over
h.recordSuccess(); p.recordRoute(connector.name(), null); p.recordFailure("DECLINED_" + resp.code);
p.transition(PaymentStatus.FAILED); publish(p); return;
case TECHNICAL_ERROR: // request not processed: safe to retry, then fail over
h.recordFailure(clock);
break;
}
}
}
p.recordFailure("ALL_ROUTES_FAILED");
p.transition(PaymentStatus.FAILED);
publish(p);
}
/** Inbound async result from the bank (UPI callback). Idempotent: duplicates/late callbacks are ignored. */
public Payment onBankCallback(String paymentId, boolean success) {
Payment p = find(paymentId);
PaymentStatus target = success ? PaymentStatus.SUCCESS : PaymentStatus.FAILED;
if (p.transitionIf(PaymentStatus.PENDING, target)) {
if (!success) p.recordFailure("BANK_CALLBACK_FAILED");
publish(p);
}
return p;
}
public Payment refund(String paymentId) {
Payment p = find(paymentId);
synchronized (p) { // per-payment monitor: prevents double refund; the lock is scoped to ONE payment
if (p.status() != PaymentStatus.SUCCESS) throw new GatewayException("Only SUCCESS payments can be refunded, was " + p.status());
BankConnector c = connectors.stream().filter(x -> x.name().equals(p.connectorName())).findFirst().orElseThrow();
if (!c.refund(p.bankRef(), p.request().amountMinor)) throw new GatewayException("Bank refused refund");
p.transition(PaymentStatus.REFUNDED);
}
publish(p);
return p;
}
public Payment find(String paymentId) {
Payment p = byId.get(paymentId);
if (p == null) throw new GatewayException("Unknown payment " + paymentId);
return p;
}
private void publish(Payment p) {
PaymentEvent e = new PaymentEvent(p);
for (PaymentListener l : listeners) {
try { l.onEvent(e); } catch (RuntimeException ignored) { /* listener failures never affect the payment */ }
}
}
}
5.7 Demo driver
// Main.java
import java.time.Clock;
import java.time.Duration;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
public class Main {
static final Map<String, String> VISA = Map.of("pan", "4111 1111 1111 1111", "cvv", "123", "expiry", "12/29");
static int n;
static PaymentRequest card(String key, String order, long amt) {
return new PaymentRequest("M1", order, key, amt, PaymentMethodType.CARD, VISA);
}
public static void main(String[] args) throws Exception {
Clock clock = Clock.systemUTC();
List<Long> sleeps = new CopyOnWriteArrayList<>();
Sleeper sleeper = sleeps::add; // record backoff instead of sleeping
ScriptedConnector hdfc = new ScriptedConnector("HDFC", Set.of(PaymentMethodType.CARD, PaymentMethodType.NET_BANKING),
() -> AuthResponse.approved("HDFC-REF"));
ScriptedConnector icici = new ScriptedConnector("ICICI", Set.of(PaymentMethodType.CARD, PaymentMethodType.NET_BANKING),
() -> AuthResponse.approved("ICICI-REF"));
ScriptedConnector npci = new ScriptedConnector("NPCI-UPI", Set.of(PaymentMethodType.UPI),
() -> AuthResponse.pending("UPI-REF"));
HealthRegistry health = new HealthRegistry(3, Duration.ofSeconds(30));
PaymentService svc = new PaymentService(PaymentMethodFactory.withDefaults(), List.of(hdfc, icici, npci),
new StaticPriorityRouting(), new RetryPolicy(2, 100), health, sleeper, clock);
// merchant webhook endpoint that fails the first two deliveries
AtomicInteger hookCalls = new AtomicInteger();
WebhookDispatcher hooks = new WebhookDispatcher(sleeper, 4);
hooks.register("M1", new WebhookDispatcher.Endpoint("https://merchant.example/hook", "whsec_demo",
(url, body, sig) -> hookCalls.incrementAndGet() > 2));
svc.addListener(hooks);
System.out.println("== 1. transient error on HDFC, retry on same connector succeeds ==");
hdfc.thenReturn(AuthResponse.technical("TIMEOUT_503"));
System.out.println(svc.pay(card("k1", "ORD-1", 49900)));
System.out.println("== 2. hard decline: no retry, no fallback ==");
hdfc.thenReturn(AuthResponse.declined("51_INSUFFICIENT_FUNDS"));
System.out.println(svc.pay(card("k2", "ORD-2", 99900)));
System.out.println("== 3. HDFC down: retries exhausted -> fallback to ICICI ==");
hdfc.thenReturn(AuthResponse.technical("503"), AuthResponse.technical("503"));
System.out.println(svc.pay(card("k3", "ORD-3", 15000)));
System.out.println("== 4. third consecutive HDFC failure (2 in step 3 + 1 now) opens the circuit ==");
hdfc.thenReturn(AuthResponse.technical("503"));
System.out.println(svc.pay(card("k4", "ORD-4", 1000)));
System.out.println(svc.pay(card("k5", "ORD-5", 2000))); // HDFC skipped entirely
System.out.println("HDFC calls so far: " + hdfc.calls());
System.out.println("== 5. smart routing uses live success rate ==");
PaymentService smart = new PaymentService(PaymentMethodFactory.withDefaults(), List.of(hdfc, icici, npci),
new SuccessRateRouting(), new RetryPolicy(2, 100), health, sleeper, clock);
System.out.println(smart.pay(card("k6", "ORD-6", 3000)));
System.out.println("== 6. idempotency ==");
int before = icici.calls() + hdfc.calls();
Payment a = svc.pay(card("k1", "ORD-1", 49900));
System.out.println("replay returns same payment: " + a.id() + " " + a.status() + ", extra bank calls = " + (icici.calls() + hdfc.calls() - before));
try { svc.pay(card("k1", "ORD-1", 1)); } catch (GatewayException e) { System.out.println("conflict: " + e.getMessage()); }
try { svc.pay(new PaymentRequest("M1", "ORD-7", "k7", 100, PaymentMethodType.CARD, Map.of("pan", "4111111111111112", "cvv", "123", "expiry", "12/29"))); }
catch (PaymentValidationException e) { System.out.println("validation: " + e.getMessage()); }
System.out.println("== 7. UPI: PENDING then async bank callback (duplicates ignored) ==");
Payment upi = svc.pay(new PaymentRequest("M1", "ORD-8", "k8", 25000, PaymentMethodType.UPI, Map.of("vpa", "alice@okbank")));
System.out.println(upi);
svc.onBankCallback(upi.id(), true);
svc.onBankCallback(upi.id(), false); // late/duplicate conflicting callback: ignored
System.out.println("after callbacks: " + upi.status());
System.out.println("== 8. refund ==");
System.out.println(svc.refund(upi.id()).status());
try { svc.refund(upi.id()); } catch (GatewayException e) { System.out.println("second refund: " + e.getMessage()); }
System.out.println("== 9. concurrency: 100 threads, same idempotency key ==");
PaymentService svc2 = new PaymentService(PaymentMethodFactory.withDefaults(), List.of(icici), new StaticPriorityRouting(),
new RetryPolicy(2, 100), new HealthRegistry(3, Duration.ofSeconds(30)), sleeper, clock);
int iciciBefore = icici.calls();
ExecutorService pool = Executors.newFixedThreadPool(16);
Set<String> ids = ConcurrentHashMap.newKeySet();
List<Future<?>> fs = new ArrayList<>();
for (int i = 0; i < 100; i++) fs.add(pool.submit(() -> ids.add(svc2.pay(card("same-key", "ORD-9", 500)).id())));
for (Future<?> f : fs) f.get();
pool.shutdown();
System.out.println("distinct payments=" + ids.size() + ", bank calls=" + (icici.calls() - iciciBefore));
System.out.println("backoff delays recorded (ms, retries + webhook redeliveries): " + sleeps.subList(0, Math.min(sleeps.size(), 6)) + " ...");
}
}
Real output (from java Main)
== 1. transient error on HDFC, retry on same connector succeeds ==
[webhook] PAY-1 SUCCESS attempt 1 failed
[webhook] PAY-1 SUCCESS attempt 2 failed
[webhook] PAY-1 SUCCESS attempt 3 delivered
PAY-1 ORD-1 SUCCESS via HDFC trail=[HDFC:TECHNICAL_ERROR, HDFC:APPROVED]
== 2. hard decline: no retry, no fallback ==
[webhook] PAY-2 FAILED attempt 1 delivered
PAY-2 ORD-2 FAILED (DECLINED_51_INSUFFICIENT_FUNDS) via HDFC trail=[HDFC:DECLINED]
== 3. HDFC down: retries exhausted -> fallback to ICICI ==
[webhook] PAY-3 SUCCESS attempt 1 delivered
PAY-3 ORD-3 SUCCESS via ICICI trail=[HDFC:TECHNICAL_ERROR, HDFC:TECHNICAL_ERROR, ICICI:APPROVED]
== 4. third consecutive HDFC failure (2 in step 3 + 1 now) opens the circuit ==
[webhook] PAY-4 SUCCESS attempt 1 delivered
PAY-4 ORD-4 SUCCESS via ICICI trail=[HDFC:TECHNICAL_ERROR, HDFC:SKIPPED_CIRCUIT_OPEN, ICICI:APPROVED]
[webhook] PAY-5 SUCCESS attempt 1 delivered
PAY-5 ORD-5 SUCCESS via ICICI trail=[HDFC:SKIPPED_CIRCUIT_OPEN, ICICI:APPROVED]
HDFC calls so far: 6
== 5. smart routing uses live success rate ==
PAY-1 ORD-6 SUCCESS via ICICI trail=[ICICI:APPROVED]
== 6. idempotency ==
replay returns same payment: PAY-1 SUCCESS, extra bank calls = 0
conflict: Idempotency key reused with a different request: k1
validation: Invalid card number
== 7. UPI: PENDING then async bank callback (duplicates ignored) ==
PAY-8 ORD-8 PENDING via NPCI-UPI trail=[NPCI-UPI:PENDING]
[webhook] PAY-8 SUCCESS attempt 1 delivered
after callbacks: SUCCESS
== 8. refund ==
[webhook] PAY-8 REFUNDED attempt 1 delivered
REFUNDED
second refund: Only SUCCESS payments can be refunded, was REFUNDED
== 9. concurrency: 100 threads, same idempotency key ==
distinct payments=1, bank calls=1
backoff delays recorded (ms, retries + webhook redeliveries): [100, 100, 200, 100] ...
Reading the output: step 5 shows the smart service (a separate instance, so ids restart at PAY-1) choosing ICICI first because HDFC's live success rate is lower. In step 4 note HDFC:TECHNICAL_ERROR, HDFC:SKIPPED_CIRCUIT_OPEN - the breaker opened mid-payment on the third consecutive failure so the second retry was skipped.
6. Walkthrough of key flows
pay() happy path with failover
Merchant PaymentService PaymentMethod Idempotency Routing HDFC ICICI Listeners
|--pay(req)-->| | | | | | |
| |--prepare(details)----->| validate, mask | | | |
| |--------------------putIfAbsent(merchant:key)-->| (null => I own it) | |
| |-transition CREATED->PROCESSING ------------------------------------------------>| event
| |--route(capable)------------------------->| [HDFC, ICICI]
| |--authorize--------------------------------------->| TECHNICAL_ERROR
| |-backoff 100ms, retry---------------------------->| TECHNICAL_ERROR
| |-(attempts exhausted: fail over)
| |--authorize----------------------------------------------->| APPROVED
| |-transition PROCESSING->SUCCESS ----------------------------------------------->| event -> webhook
|<--Payment---|
UPI async
pay() -> connector returns PENDING -> state PENDING (HTTP 202 to merchant)
bank/NPCI POSTs callback -> onBankCallback(id, success)
payment.transitionIf(PENDING, SUCCESS|FAILED) // atomic; false => duplicate/late, ignored
-> publish event -> webhook to merchant
Webhook delivery
event(SUCCESS) -> body JSON + HMAC_SHA256(secret, body) in header -> POST
fail -> sleep 100ms -> retry -> sleep 200ms -> retry ... up to N -> dead-letter queue (manual/automatic replay)
merchant verifies signature and dedupes on paymentId+status (at-least-once delivery)
Retry/failover decision table (the core insight)
| Bank answer | Meaning | Action |
|---|---|---|
| APPROVED | money moved | SUCCESS, stop |
| DECLINED (insufficient funds, wrong CVV, blocked card) | business decision, same on any bank | FAILED, no retry, no failover (would annoy customer/risk fraud flags) |
| TECHNICAL_ERROR (connect refused, 503 before processing) | request not processed | retry w/ backoff, then fail over |
| PENDING | accepted, result later | PENDING, wait for callback / poll status |
| TIMEOUT (not modelled) | unknown whether processed | never blind retry: status-inquiry/reverse first (see follow-ups) |
7. Extensibility
- New payment method (BNPL, EMI, crypto): implement
PaymentMethod,factory.register(...), add enum constant, make a connector reportsupports(...).PaymentServiceunchanged. - New bank/acquirer: implement
BankConnector(Adapter) and add to the list. Capability matching is bysupports(). - New routing rule (by card network, amount slab, MDR cost, merchant preference, A/B experiment): implement
RoutingStrategy; compose with a decoratornew CostAwareRouting(new SuccessRateRouting()). Instrument carriesnetworkandbankattributes for this purpose. - New listener (fraud analytics, ledger posting, email):
addListener. - New retry policy (jitter, per-method budgets, retry budget per merchant): swap
RetryPolicy. - New terminal/intermediate status (AUTHORIZED -> CAPTURED for auth/capture flows, PARTIALLY_REFUNDED): add to
PaymentStatus.next(); other code only reacts to statuses it cares about.
8. Concurrency and edge cases
- Idempotency:
putIfAbsentonmerchantId:key-> exactly one thread runs the payment; duplicates receive the samePayment(demo: 100 threads, 1 payment, 1 bank call). Key scoped per merchant. Fingerprint mismatch -> conflict error. In production persist the key with a UNIQUE constraint and TTL. - Lock granularity: per-
Paymentmonitor for transitions (transition,transitionIf); everything else is lock-free (ConcurrentHashMap, atomics inConnectorHealth). Unrelated payments never contend; a global lock would serialise the gateway. - Callbacks race with pay():
transitionIf(PENDING, ...)is a CAS-style transition: duplicates, conflicting late callbacks, or a callback arriving before PENDING was recorded (it simply does nothing; the reconciler/poller resolves). - Refund holds the payment's own monitor while calling the bank to prevent double refund; trade-off: IO under a lock, scoped to a single payment. Production alternative: add
REFUND_INITIATEDstate and CAS into it, then call the bank outside the lock. - Listeners run outside locks and exceptions are swallowed: a dead merchant endpoint must never fail a payment. In production publish to a queue/outbox and let workers deliver (the sync loop here would block the request thread).
- Retry storms: exponential backoff + jitter + per-connector circuit breaker + global retry budget.
- Edge cases: invalid PAN (Luhn), expired card, amount <= 0, unsupported method, no capable connector (-> FAILED
ALL_ROUTES_FAILED), all circuits open, duplicate callbacks, callback for unknown payment, merchant without webhook endpoint, clock skew on breaker (injectedClock), webhook signature replay (add timestamp to signed body).
9. Follow-up questions and model answers
- How do you avoid a double charge when the bank call times out? A timeout is unknown. Do not retry the authorize blindly. Mark the payment PENDING, call the bank's status inquiry API (with our reference), or send a reversal/void (idempotent by our txn id) before trying another bank; a reconciliation job closes anything still PENDING after N minutes using the bank's settlement/report files. Also send a stable
paymentIdto the bank as the merchant reference so the bank can dedupe. - Why not fail over on a decline? A decline is the issuer's decision (insufficient funds, risk). Another acquirer gets the same answer; extra attempts increase fraud-score risk and may double-hold funds.
- How do you persist this?
payment(id, merchant_id, order_id, idempotency_key, request_hash, amount, method, status, connector, bank_ref, version, created_at)withUNIQUE(merchant_id, idempotency_key);payment_attempt(payment_id, seq, connector, outcome, code, latency_ms, ts);webhook_delivery(id, payment_id, status, attempts, next_retry_at);outbox(event). State changes use optimistic locking:UPDATE payment SET status=? , version=version+1 WHERE id=? AND status=?(rows==1) - this is the DB form oftransitionIf. - Make it distributed / horizontally scalable? Stateless gateway nodes behind a load balancer; idempotency in DB/Redis (
SET NX+ DB unique constraint); circuit-breaker state local per node (cheap, converges) or shared via Redis for global view; success-rate metrics from a streaming aggregation (sliding window) feeding the router; callbacks routed to any node (they act through the DB CAS). Partition payments bymerchantIdorpaymentIdfor hot-key isolation. - Exactly-once webhooks? Impossible; provide at-least-once with a stable event id and signature; merchants dedupe. Use transactional outbox so an event exists iff the state change committed.
- How is the routing decision made smarter? Sliding-window success rate per (connector, method, issuer bank/BIN, amount band); cost (MDR) weighted score; exploration (epsilon-greedy / 5% traffic) so a recovering bank gets sampled; manual overrides per merchant; kill switches.
- Authorize vs capture? Add AUTHORIZED state and a
captureoperation with expiry; refunds partial amounts tracked by arefunded_amountand a cap check under CAS. - Security? Never persist PAN/CVV (PCI-DSS); tokenise via a vault; TLS + mTLS to banks; HMAC-signed webhooks with timestamp; API auth per merchant; mask in logs (
Instrument.maskedRef). - How to test? Scripted connectors (as in the demo), injected
ClockandSleeper, property test on the state machine (random event sequences never reach illegal states), concurrency test on same key. - Where do wallets/ledger fit? On SUCCESS, a listener posts a ledger entry (merchant payable, MDR fee) - the double-entry design from the wallet problem; settlement later pays the merchant.
10. Common mistakes, red flags, rubric
Red flags
- One giant
processPayment()withif (method == CARD) ... else if (UPI). - Retrying everything, including declines; failing over after an ambiguous timeout without status inquiry.
- No idempotency, or idempotency with
containsKey/putrace; no payload-mismatch check. - Status as free
String/boolean flags; allowing SUCCESS -> FAILED. - Doing webhook HTTP inside the locked section / failing the payment when the webhook fails.
- Storing raw card number/CVV; logging full PAN.
Thread.sleephard-wired (untestable); no backoff or infinite retry loops.- Assuming synchronous bank responses only (no PENDING/callback story for UPI).
Mini rubric (25 pts)
| Area | Points |
|---|---|
| Requirements: async, idempotency, retries vs declines clarified | 3 |
| Strategy/Factory for methods; Adapter for banks | 4 |
| Routing + retry + fallback semantics (decline vs technical), circuit breaker | 5 |
| State machine with guarded transitions + callbacks | 4 |
| Idempotency and thread-safety reasoning (lock granularity) | 4 |
| Webhooks via Observer: signing, retry, DLQ | 2 |
| Working demo + persistence/distributed follow-ups | 3 |