Search

Lessons (40)

• 500 Users Upload Documents at Once: Design the Background Job Pipeline

  Move slow document processing out of the HTTP request with object storage, a job table, a task queue …

• StatefulSets, DaemonSets, Jobs and CronJobs

  … Limit it with nodeSelector if only some nodes need it. Job: run to completion yaml apiVersion: batch/v1 …

• Go Concurrency: Goroutines, Channels and Context

  … limit how much runs at once go func process(ctx context.Context, jobs []Job, workers int) []Result { jobCh …

• Timeouts, Retries & Backpressure

  … ten tiny jobs and ten huge jobs look equal. Track oldest job age and estimated drain time too …

• Concurrency, Resilience & Distributed Workflows

  … A production job should have: - a durable queue and a unique job ID; - a visibility timeout or lease …

• CI/CD and Reviewed Production Changes

  … Two jobs writing the same state should not race; use both backend locking and CI concurrency controls. Commands …

• AWS Messaging: Queues, Topics and Events

  … For example, after five failed receives, move the problematic message aside so workers can continue with other jobs …

• L6 Production Debugging Casebook

  … Check cron jobs, Kubernetes Jobs, @Scheduled jobs, report generation, backups, autoscaling events, cache TTL cohorts and deployment history …

• Message Queues & RabbitMQ

  … each message → ONE consumer"] Pr1[Producer] -- Q1[(jobs)] -- C1[Worker A] Q1 -- C2[Worker B] end subgraph PS …

• Small Functions That Do One Thing

  … In a professional kitchen, chefs use separate tools, and each one is excellent at a single job. ![One …

• The Hot SKU Alert Problem

  … Create a durable fan-out job and split watchers into resumable ranges. Product ordering and delivery parallelism are …

• Flash Sale: 10 Million Buyers, No Oversell

  … On success, reservation becomes SOLD ; on expiry, a single guarded job returns stock only if it is still …

• AWS Data Pipelines in Plain Language

  … A Glue job runs transformations, such as converting raw CSV into typed, validated Parquet. A crawler can help …

• Kubernetes Cluster Architecture, Simply

  … Knowing their jobs helps when you read errors or diagrams. Name Plain-English job --- --- API server The front …

• Batch vs Stream and Data Modeling

  … Build reporting tables in scheduled batches or incremental jobs. Denormalize read models deliberately and retain lineage back to …

• Deployments: Keep Your App Running

  … log agent Job Work that should finish once, such as a migration CronJob A Job on a schedule …

• Service Communication: Sync vs Async

  … sync for queries the user waits on, events for state changes, queues for background jobs.

• AWS: Start Here

  … On AWS, choose a service for each job: Part of the shop Example AWS choice Why it exists …

• S — Single Responsibility Principle

  … br/ cooks + serves + bills"] end subgraph Good["✅ One job each"] C["👨‍🍳 Chef<br/ cooks"] -- W["🧑‍💼 Waiter<br …

• React vs Vue vs Angular

  … Angular SSR / Analog Learning curve Medium Gentle Steeper Job market Largest Strong (esp. Asia and Europe) Strong in …

• Heaps & Priority Queues

  … schedulers: run the job with the nearest deadline first (OS schedulers, BullMQ delayed jobs). - Dijkstra's shortest path …

• Layered (N-Tier) Architecture

  … each layer has one job and only talks to the layer directly below it. ![Layered architecture: requests flow …

• Choosing a Programming Language

  … attachment for every job, especially data and AI. - A nail gun (Go) does one job, network services, very …

• Microkernel (Plugin) Architecture

  … parse to HTML (core job) Core- L: afterRender(html) L-- Core: html with safe links Core-- App: final …

• Serverless Architecture

  … JSON.stringify(result) }; }; When to use it ✅ Spiky or unpredictable traffic, event processing (uploads, queues, webhooks), scheduled jobs …

• Edge, Platform & Frontend Delivery

  … CDN, edge, server and API layers have different jobs and trust boundaries. - Middleware should stay small and cannot …

• Architecture Patterns at Scale

  … effects/read models Duplicate, late event handling Serverless job Spiky isolated work Limits, cold starts, long workflows For …

• Backend Design Scenarios

  … Separate campaign submission from delivery: mermaid flowchart LR A[Campaign API] -- J[(Durable jobs)] J -- P[Partition by …

• Environments, Workspaces, and Multiple Accounts

  … Which control stops a dev CI job from reading production state? Further reading: [CLI workspaces](https://developer.hashicorp …

• Design a Reliable Notification System

  … Watch oldest-job age, provider errors, retry rate, opt-out enforcement and tenant backlog. A notification is not …

• Distributed Coordination: Consensus, Heartbeats and Gossip

  … smallest tool Need Usually enough --- --- Run one periodic job Database claim with lease + fencing. Discover healthy peers Heartbeats …

• Proxies, APIs and Real-Time Protocols

  … return a job ID and process it through a queue rather than hold an HTTP request open. Real …

• SOLID at a Glance

  … they connect mermaid flowchart TB S["S — one job per class"] -- Small[Small, focused pieces] I["I — small …

• Drift, Troubleshooting, and Disaster Recovery

  … Explain why force-unlocking immediately is unsafe if the original job is still running. Further reading: [State management …

• Message Delivery: Queues, Kafka, Ordering and DLQs

  … queue usually means "one worker should handle this job." Kafka is a durable, partitioned event log: several consumer …

• I — Interface Segregation Principle

  … the fields they need. - Repository interfaces: a report job gets a ReadOnlyOrderRepository , not one with delete() . ts interface …

• Observability and Incident Response

  … one propagated context through HTTP, queues and background jobs. - Logs: structured, searchable events with no secrets or unnecessary …

• Design a URL Shortener

  … It should not wait for analytics, fraud dashboards, emails or aggregation jobs. Choosing a short code Base62 ( a …

• Resilient Service Calls: Retries, Breakers and Load Shedding

  … async work Delay receipts, but never store infinite jobs in memory. The retry rule Retrying a non-idempotent …

• KISS: Keep It Simple, Stupid

  … A light switch does the same job in one step. ![KISS: a light switch beats a Rube Goldberg …