36 questions
DevOps interview questions and answers
Junior questions
1. Postgres for local development runs in Docker. After
docker rmand a newdocker runwith the same image, all the data is gone, although a colleague says 'but the image is the same'. What did he miss, and how do you run the database so data survives recreating the container?Answer
An image is an immutable template made of layers, and a container is a running instance with its own thin writable layer, which disappears with
docker rm. The official Postgres image declares aVOLUMEfor the data directory, so Docker created an anonymous volume with a random name; the new container got a new empty volume, while the old one was left dangling, and the data can still be found withdocker volume ls. The right way: a named volume mounted at the data directory, which survives recreating the container, or a bind mount to a host folder. In compose this is thevolumessection, and remember thatdocker compose down -vanddocker volume prunedelete those too.Follow-up question: How do you back up this volume without stopping development for long?
- Course: Docker basics
- Knowledge base: Docker
- Source: docs.docker.com
2. A team of five merges straight into main, and every Monday main is broken: someone's types fail, someone's tests fail, and the author says 'it worked on my machine'. What minimal CI do you set up, what does it run, and what blocks a merge?
Answer
A check on every pull request in a clean environment: install dependencies strictly from the lockfile (
pnpm install --frozen-lockfileornpm ci), lint, type check, tests and build, on the same runtime version as production. Protect the main branch: merge only through a PR with required green checks, ideally also requiring the branch to be up to date with main. 'It worked on my machine' usually means different dependency versions, uncommitted files or local environment variables, and a clean CI environment is exactly what catches that. The pipeline must be fast, a few minutes, with a dependency cache, otherwise people start bypassing it.Follow-up question: Each PR is green on its own, yet after two PRs merge main is red. How does that happen, and what do you do about it?
- Course: CI/CD with GitHub Actions
- Knowledge base: GitHub Actions
3. An app in a container listens on
localhost:3000, the container runs with-p 3000:3000, the logs say the server started, butcurl localhost:3000from the host givesconnection reset. Why, and how do you fix it?Answer
A container has its own network namespace, and
localhostinside it is the container's own loopback, not the host's. Docker forwards the host port to the container's network interface, and a server listening only on127.0.0.1does not accept connections on that interface. The fix: listen on0.0.0.0, for exampleapp.listen(3000, '0.0.0.0')orHOST=0.0.0.0for dev servers that bind to localhost by default. You can check the hypothesis withdocker execandcurlinside the container: there the request succeeds, which confirms the problem is the listen address, not the app.Follow-up question: How does the container reach a database running on the host, and how does it reach another container in the same compose project?
- Course: Docker basics
- Knowledge base: Docker
4. You deployed a new version and the pod is in
CrashLoopBackOff. What do you check first, and in what order?Answer
CrashLoopBackOffmeans the container starts and exits, and the kubelet restarts it with an increasing delay. First:kubectl describe pod(events,Last State, the exit code and reason, for exampleOOMKilledwith code 137 or an error in the start command) andkubectl logs --previous, because the current container may not have written anything yet. Then the usual causes: the app dies at startup because of a missing environment variable, Secret or ConfigMap, an unreachable database or a migration; the memory limit is too small; a liveness probe kills a slow start; a wrong entrypoint or image architecture. If the old version worked, compare the manifests and roll back quickly withkubectl rollout undo, then investigate without pressure.Follow-up question: How do you tell that the OOM killer killed the container, rather than the app exiting with an error?
- Course: Kubernetes: the basics
- Knowledge base: Kubernetes
- Source: kubernetes.io
5. A colleague builds two different Docker images, one for staging and one for prod, and copies a separate
.envwith the database address and the Stripe key into each. What is wrong with this, and how do you do it properly?Answer
There should be one immutable image: whatever passed the checks on staging is what goes to prod, and two images mean prod gets an artifact nobody tested. On top of that, secrets inside an image are visible to anyone who can pull it from the registry, they stay in the layers and in the build cache, and rotating a key requires a rebuild. The 12 factor way: config and secrets come from the environment at run time (environment variables, a Kubernetes Secret, Docker secrets, a secrets manager such as Vault or AWS Secrets Manager), and
.envgoes into.dockerignoreand.gitignore. The app validates required variables at startup and fails with a clear error if something is missing.Follow-up question: The Stripe key has already made it into an image in the registry. What do you do?
- Course: Docker and deployment
- Knowledge base: Docker
6. Users report that checkout sometimes takes 10 seconds. You have logs, Prometheus metrics and OpenTelemetry traces. Where do you start, and what does each of these signals tell you?
Answer
Metrics answer 'how often and how much': the latency histogram of the checkout endpoint shows whether it is 1 % or 30 % of requests, since when, and whether it lines up with a deploy or with load. Traces answer 'where exactly': I open a trace of a slow request and see which span takes 9 seconds, for example the payment provider call or a database query. Logs give the details of a specific event: the error text, the parameters, and I find them by the
trace_idfrom the trace. So the order is metrics, then traces, then logs, and all three must be linked by a shared trace id.Follow-up question: Why might the average latency on the dashboard not show this problem at all?
- Knowledge base: Prometheus, OpenTelemetry, Grafana Loki
7. In docker compose the API fails with
connection refusedto Postgres even thoughdepends_onis set. Why, and how do you fix it?Answer
A plain
depends_ononly controls container start order, not service readiness. The Postgres container is already running, but the database is still initializing. The solution: ahealthcheckon the database service (for examplepg_isready) anddepends_onwith theservice_healthycondition. Also the app should retry connecting with backoff, because the database can restart in production too.Follow-up question: How does liveness differ from readiness?
- Course: Docker and deployment
- Knowledge base: Docker
- Source: docs.docker.com
Middle questions
1. The product lets customers upload their own plugins, and their code runs on your servers. The team suggests simply running each plugin in its own Docker container. Is that enough isolation, and what do you propose?
Answer
A container shares the kernel with the host and is isolated by kernel features: namespaces limit what a process sees, cgroups limit resources, seccomp and capabilities limit system calls. For your own code that is enough, but for someone else's untrusted code the kernel becomes a shared attack surface: one kernel vulnerability or a config mistake (
--privileged, a mounteddocker.sock, root inside) gives a way out to the host and to other customers. A virtual machine has its own kernel on top of a hypervisor, so the isolation boundary is much narrower, but it is heavier. The compromise for this case: a microVM such as Firecracker or Kata, or the gVisor sandbox runtime, which start almost like a container but do not expose the host kernel directly, plus non root, a restricted network, CPU and memory limits and timeouts.Follow-up question: How do you restrict where in the network the plugin code can connect?
- Course: Docker basics
- Knowledge base: Docker
2. The team wants every merge to main to go to production automatically, with no button. What must be true before you agree?
Answer
This is the move from Continuous Delivery, where the artifact is always ready and a human triggers the deploy, to Continuous Deployment, where there is no manual step. Prerequisites: tests the team genuinely trusts, since they are now the only gate; progressive rollout (canary or staged) with automatic checks of error rate and latency and automatic rollback; a fast, rehearsed rollback; backward compatible database migrations following expand and contract; feature flags to separate shipping code from turning a feature on; and observability that shows a problem within minutes. Without these, automatic deployment just delivers bugs to users faster.
Follow-up question: How do you handle a release that must go live at the same moment as a marketing campaign?
- Course: CI/CD with GitHub Actions
- Knowledge base: GitHub Actions
3. The new version's pods are
RunningandReady, but requests through the Service getconnection refusedand the Ingress returns 503. What do you check?Answer
The chain is: the Ingress controller sends traffic to the Service, the Service finds pods by its
selectorand forwards to thetargetPort. First:kubectl get endpointslicesfor that Service: if there are no addresses, theselectordoes not match the pod labels, for example after a label was renamed in the new manifest. If endpoints exist, checktargetPort: it must match the port the container actually listens on, and the app must listen on0.0.0.0. Then the Ingress: the right Service name and port inbackend, theingressClassName, and the controller logs. It is convenient to find the break layer by layer withkubectl port-forward, first to the pod, then to the Service.Follow-up question: Why is a separate LoadBalancer Service per service usually a worse choice than a single Ingress?
- Course: Kubernetes: the basics
- Knowledge base: Kubernetes, nginx
- Source: kubernetes.io
4. A Node app image weighs over a gigabyte, and every one-line code change reinstalls all dependencies. How do you fix the Dockerfile?
Answer
Each instruction makes a layer, and a changed layer invalidates the cache of the following ones. If
COPY . .comes beforenpm install, any code edit reinstalls dependencies. First copypackage.jsonwith the lock file, runnpm ci, and only then the rest of the code. For size, use multi-stage: build in one stage, and only the built code and production dependencies reach the final slim image viaCOPY --from. Also add.dockerignore, run as a non-root user and pin the image tag instead oflatest.Follow-up question: Why is
npm cibetter thannpm installin a Dockerfile?- Course: Docker basics
- Knowledge base: Docker
- Source: docs.docker.com
5. A Dockerfile copies a token into the image and the next command deletes the file with
rm. Is the token gone, and how should you pass a secret into a build correctly?Answer
It is not gone. Layers are only added:
rmcreates a new layer with a deletion mark, while theCOPYlayer stays in the image and goes to the registry. Anyone who can pull will unpack the layer and read the token. The right way is a BuildKit secret:RUN --mount=type=secret,id=npmrctogether withdocker build --secret, so the secret is available only during that command and does not land in a layer.ARGis not suitable either, because values stay in the image history.Follow-up question: How do you make sure a finished image contains no secrets?
- Course: Docker basics
- Knowledge base: Docker
- Source: docs.docker.com
6.
docker stopon your Node container always waits 10 seconds and then kills it. Why, and what do you do about it?Answer
docker stopsendsSIGTERMto the process with PID 1 andSIGKILLafter the timeout. A PID 1 process in Linux has no default signal handlers, so if the app does not listen forSIGTERM, it does not exit. A common cause is also starting throughnpm startor the shell form ofCMD: the shell gets the signal, not Node. Fix: exec formCMD ["node", "dist/main.js"], aSIGTERMhandler that closes the server and connections, and if needed the--initflag or tini.Follow-up question: What should graceful shutdown do in an HTTP server?
- Course: Docker and deployment
- Knowledge base: Docker
7. Every rolling deploy causes 30 seconds of 5xx: some requests fail with 502 and
connection refused. Probes are configured and the image is fine. What is happening, and how do you get rid of the errors?Answer
The errors come from both sides. A new pod gets traffic as soon as its readiness probe passes, and if the probe only checks that the process is alive rather than that the app has warmed up its connections and is ready, the first requests fail; readiness must reflect real readiness, and a startupProbe covers a slow start. An old pod being terminated gets SIGTERM at the same time as it is removed from the endpoints, but kube-proxy and the Ingress learn about it with a delay and keep sending traffic for a few seconds to a process that has already shut down. The fix: a
preStophook with a pause of a few seconds, SIGTERM handling with graceful shutdown (stop accepting new connections, finish in flight requests),terminationGracePeriodSecondslonger than that, andmaxUnavailable: 0in the strategy. Liveness must not check the database, otherwise a database outage restarts every pod at once.Follow-up question: How do you verify that graceful shutdown really works before users notice?
- Course: Kubernetes: the basics
- Knowledge base: Kubernetes
- Source: kubernetes.io
8. A Node service in Kubernetes restarts every few hours with
OOMKilled, exit code 137, although the logs show no errors and the heap graph shows only 300 MB. The memory limit is 512Mi. Why, and what do you change?Answer
Exit code 137 means the process was killed with SIGKILL, and
OOMKilledsays the kernel did it because the container exceeded the memory limit of its cgroup. The limit counts all of the process memory, not just the heap: buffers, native modules, thread stacks, code and part of the page cache, so RSS can be much larger than the heap. On top of that, the V8 heap size is not always aligned with the container limit, and the heap can keep growing until the container is killed. What to do: monitor RSS andcontainer_memory_working_set_bytes, not only the heap; set--max-old-space-sizeto roughly 75 % of the limit so V8 collects garbage earlier; set the memory request equal to the limit so the pod is not evicted under node pressure; and look for a leak if RSS grows steadily.Follow-up question: Another service has p99 spikes while CPU on the graph is only 40 % of its limit. What could be causing it?
- Course: Kubernetes: the basics
- Knowledge base: Kubernetes
9. The team keeps the database password in a Kubernetes Secret and believes it is encrypted. A developer with read access to the namespace pulled it out in a minute with
kubectl get secret -o yaml. What does a Secret really give you, and how do you protect the data?Answer
By default a Secret is stored in etcd only as base64, which means unencrypted, and anyone with
geton secrets in the namespace, or who can create a pod that mounts it, can read the value. Protection comes in layers: RBAC that grants access to secrets only to the service accounts that use them and not to people; encrypting etcd at rest with anEncryptionConfigurationor KMS; an external secrets manager (Vault, AWS Secrets Manager) with the External Secrets Operator or the CSI driver, so the cluster is not the source of truth; and no plain secrets in Git, only ones encrypted with Sealed Secrets or SOPS. Separately: mount secrets as files rather than environment variables, because variables are easier to leak into logs or a dump by accident.Follow-up question: The password must be rotated every month without downtime. How do you set that up?
- Course: Kubernetes: the basics
- Knowledge base: Kubernetes, Helm
- Source: kubernetes.io
10. Describe a pipeline for a backend service. What are the stages, how does cache differ from artifacts, and how do you protect the deploy?
Answer
Typical stages: install dependencies, linter and types, unit tests, image build, integration tests, push to the registry, deploy. A cache speeds up repeated runs (for example downloaded dependencies) and can be lost without harm. An artifact is a build result passed to later stages, and the release uses exactly that so nothing is rebuilt. The deploy is protected by: secrets only in protected variables, production deploy only from the main branch, manual approval or environment protection, and minimal token permissions.
Follow-up question: How do you make sure a pipeline running on a PR cannot read production secrets?
- Course: CI/CD with GitHub Actions
- Knowledge base: GitHub Actions, GitLab CI/CD
- Source: docs.github.com
11. In a public repository a deploy token showed up in plain text in a GitHub Actions log: a step printed it as base64 and masking did not catch it. What do you do now, and what do you change in the workflow?
Answer
Now: treat the token as compromised, revoke and reissue it, check the audit logs for use during the exposure, delete the run log. Masking works only for the exact secret value, so any transformation (base64, part of a string, JSON) bypasses it, and secrets must not reach the output at all. Changes to the workflow: OIDC instead of long lived cloud tokens, minimal
permissionsforGITHUB_TOKEN, secrets only in the jobs and environments that need them, with required reviewers for prod, nopull_request_targetthat checks out fork code, and third party actions pinned to a commit SHA rather than a tag. Plus secret scanning with push protection in the repository.Follow-up question: Why can an action pinned to the
v4tag suddenly start stealing secrets?- Course: CI/CD with GitHub Actions
- Knowledge base: GitHub Actions
12. At two in the morning an engineer fixed a security group by hand in the AWS console to bring prod back. The next morning a colleague ran
terraform applyfor a different change, the fix was silently reverted and the incident repeated. Why did that happen, and how do you prevent it?Answer
Terraform compares the desired state in code with the real resources and the state, and sees any manual change as drift to be removed, so
applyrestored the security group to what the code says. It is a process problem, not a tool problem: a manual fix during an incident is fine, but it must go into code right away. Protection: always readterraform planbefore applying and apply only a saved plan (terraform plan -out, then apply that file) through CI with review; run plan regularly to detect drift and alert on it; use remote state with locking so two applies never run at once; and restrict manual changes to production resources in the console.Follow-up question: How do you bring resources that were created by hand and already run in production under Terraform?
- Knowledge base: Terraform, OpenTofu
- Source: developer.hashicorp.com
13. A release changes how the cart price is calculated. Last time a bad release reached every user within three minutes through a rolling deploy. Which rollout strategy do you pick now, and which metrics decide whether to proceed or roll back?
Answer
Rolling replaces pods gradually but does not check quality: if the pods pass readiness, the release reaches 100 %. Blue green gives an instant switch and an equally instant rollback, but every user gets the new version at once, and you need double capacity. For risky business logic canary is better: 1 %, then 5 %, then 25 % of traffic with pauses and automated analysis (Argo Rollouts or Flagger) comparing the canary with the baseline on 5xx rate, p99 latency and a business metric such as cart conversion or average order value. Even better, put the pricing logic behind a feature flag so it can be switched off without a deploy. Schema changes must be compatible with both versions, since they run at the same time.
Follow-up question: How do you choose who goes into the canary so a bug does not hit the most important customers?
- Course: Kubernetes: the basics
- Knowledge base: Kubernetes, Argo CD
14. How do you organize dev, staging and prod, where do config and secrets live, and what do you do when a key was committed to the repo?
Answer
Environments differ only in configuration, the image is the same. Staging is as close to prod as possible in versions and topology, but with different data and keys. Secrets live in a secrets manager or encrypted CI variables, not in the repo. If a key was committed, revoke and rotate it immediately: rewriting history does not help, because it may already be copied. Then check the key's usage logs and add a secret scanner to pre-commit and CI.
Follow-up question: How do you prevent secrets from being committed again automatically?
- Course: Docker and deployment
- Knowledge base: Terraform
15. A new checkout API launches in a month, and before launch you have to set up dashboards and on call. Which metrics do you collect for the service, which for resources, what SLO do you propose, and what exactly wakes a person at night?
Answer
For the service, RED: rate (requests per second), errors (error ratio) and duration (a latency histogram) per endpoint. For resources, USE: utilization, saturation and errors for CPU, memory, the database connection pool, queues. I phrase the SLI from the user's side, for example the share of successful checkout requests faster than 500 ms, and the SLO is a target over a window, say 99.5 % over 28 days, which gives an error budget of 0.5 % of requests to spend on releases and failures. Only what threatens the SLO pages someone, meaning a fast budget burn, not high CPU or a single error; everything else becomes a ticket for working hours.
Follow-up question: Who decides what happens when the error budget is gone in the first week?
- Knowledge base: Prometheus, Grafana
- Source: sre.google
16. On the dashboard, API p99 latency is computed as the average of each pod's p99 reported by a Summary. The graph is flat, yet users complain about slowness. What is wrong, and what should you use instead?
Answer
Quantiles cannot be averaged: the average of ten pods' p99 is not the p99 of the total traffic, and it hides a pod whose tail is much worse. A Summary computes quantiles on the client, inside the process, so they cannot be aggregated correctly across pods. A Histogram stores counters per bucket (
_bucketwith thelelabel), which can be summed across pods before computing the quantile:histogram_quantile(0.99, sum by (le) (rate(http_request_duration_seconds_bucket[5m]))). The cost is that accuracy depends on the bucket boundaries, so they must be chosen around the SLO targets, and every bucket is a separate time series.Follow-up question: How do you choose bucket boundaries if the SLO says '95 % of requests are faster than 300 ms'?
- Knowledge base: Prometheus
- Source: prometheus.io
17. A team has 200 alerts a day, everything is red, and nobody reacted to a real incident for 40 minutes. What do you do?
Answer
This is alert fatigue: people stop trusting the noise. Every alert should require human action right now, otherwise it is deleted or turned into a dashboard or ticket. Alert on user-facing symptoms (errors, latency, SLO), not on every cause. Group and deduplicate in Alertmanager so a host failure does not produce 140 messages. Add a runbook with steps to each alert. Regularly review which alerts fired for nothing and clean them up.
Follow-up question: How do you make sure one node failure does not produce a hundred alerts?
- Knowledge base: Prometheus, Grafana
18. How do you set up logging so that one
requestIdortrace_idfinds all lines of a request across all services?Answer
Logs are written in a structured form (JSON), not free text. At the request entry you take the identifier from a header such as
traceparent, or generate a new one, put it into the context (in Node, AsyncLocalStorage) and add it to every log line automatically. All outgoing calls pass the same identifier on in headers. Then in Loki or Elasticsearch one search by that field shows the whole path of the request. Secrets and personal data do not get into logs, they are masked.Follow-up question: What is W3C Trace Context and why the
traceparentheader?- Knowledge base: Grafana Loki, ELK Stack, OpenTelemetry
19. The payment service started responding in 30 seconds and the whole API went down. Why is a slow dependency worse than a dead one, and what helps?
Answer
A dead dependency gives a fast error, while a slow one holds connections, threads and memory until the queue of requests exhausts the resources of the whole service. Protection: short timeouts on all calls, a circuit breaker that opens after a series of failures and fails fast instead of waiting, limiting concurrent calls (bulkhead), and a fallback or degraded functionality. After a pause the breaker lets trial requests through and closes if the dependency recovered.
Follow-up question: How do you choose a timeout value for a call to another service?
- Knowledge base: OpenTelemetry
20. An API runs on a single server. You need 99.9% availability and five times the load headroom. What do you change?
Answer
Remove the single point of failure: several instances behind a load balancer with health checks, across zones or hosts. The app must be stateless: sessions in Redis or JWT, files in object storage rather than on the instance disk. The database becomes the next bottleneck: read replicas, automatic failover, backups. Add autoscaling on a metric, a cache for hot reads and verify everything with a load test.
Follow-up question: Why might a user be randomly logged out after moving to 3 instances?
- Knowledge base: Kubernetes, Redis
21. A nightly
pg_dumpruns. At 14:03 someone ranDELETEwithoutWHERE. What will such a backup achieve and what would help better?Answer
The nightly dump restores the state at 00:00, so all changes over 14 hours are lost. It is better to have continuous WAL archiving together with base backups: then you can do point-in-time recovery and restore to 14:02. Restoration is done on a separate instance, from which the needed data is returned to prod. A backup without a tested restore guarantees nothing, so restores are practiced regularly and RTO and RPO are measured. Copies are kept separate from the main server.
Follow-up question: What are RPO and RTO and how do they affect the choice of backup strategy?
- Knowledge base: PostgreSQL
- Source: postgresql.org
22. A pentest report on your service: the container runs as root, the image is based on full Ubuntu with curl, gcc and git,
/var/run/docker.sockis mounted into the container, and the base image has not been updated for a year. Prioritise the fixes and explain.Answer
First: remove
docker.sock, because access to it is effectively root on the host, and any RCE in the app becomes a takeover of the server. Second: run as non root (USER nodeor a numeric UID), plus--cap-drop ALL,no-new-privilegesand a read only root filesystem, to limit what an attacker can do after getting in. Third: a minimal image via a multi stage build (slim, distroless or alpine) with no compiler, utilities or git, meaning fewer vulnerabilities and fewer tools for an attacker. Fourth: regular base image updates and scanning (Trivy, Grype, Docker Scout) in CI that blocks critical CVEs. The order is set by how much each item widens the impact of a compromise.Follow-up question: CI needs Docker access to build images. How do you do it without docker.sock?
- Course: Docker and deployment
- Knowledge base: Docker
- Source: docs.docker.com
23. A release at 17:00 broke checkout. The rollback took 40 minutes: CI rebuilt the old commit and ran the tests again, and some dependencies had been updated in the meantime. How do you set up releases so a rollback takes a minute?
Answer
A rollback must not build anything. Every build produces an immutable artifact: an image tagged by commit SHA or version, not
latest, built from the lockfile, so the previous version already sits in the registry and has been tested. A rollback is switching to the previous image:kubectl rollout undo, restoring the previous tag in a GitOps repository, or flipping blue green, and it must be one rehearsed command or button. Database migrations are made backward compatible with expand and contract, otherwise the old code will not work with the new schema. Risky features go behind feature flags, and then the fastest rollback is turning off the flag without a deploy.Follow-up question: How often should you rehearse a rollback, and how do you check that it works?
- Course: Kubernetes: the basics
- Knowledge base: Kubernetes, GitHub Actions
Senior questions
1. How do you ship a new API version without downtime if the release also changes the database schema?
Answer
Migrations follow the expand and contract style. First add new columns or tables that do not break the old code version. Then ship code that writes to both places and reads from the new one, and backfill data with a background script. Only when no old version is running, drop the old columns in a separate release. The code itself is rolled out with rolling or canary, with readiness probes and graceful shutdown. This way even a failed release can be rolled back without rolling back the database.
Follow-up question: Why is renaming a column in one step dangerous?
- Course: Kubernetes: the basics
- Knowledge base: Kubernetes, PostgreSQL
2. After a release Prometheus started crashing with OOM, and a label with values like
user_idappeared in the metrics. What happened and what do you do?Answer
Each unique combination of label values is a separate time series. A label with unbounded values, such as
user_id, a URL with an id ortrace_id, creates millions of series, and Prometheus runs out of memory. Fix: remove that label from the metric, normalize the path to a route template, and keep individual identifiers in logs and traces. For diagnosis, look at the top metrics by series count, and addsample_limitto the config as a safeguard. Cardinality is also critical for Loki and other label-based systems.Follow-up question: Why is the same dangerous with labels in Loki?
- Knowledge base: Prometheus, Grafana Loki
3. SLO 99.9% over 30 days. The alert
error_ratio > 0.001 for 5mwakes people at night over every blip. How do you redo it?Answer
A simple threshold does not account for how much error budget is actually being spent. Burn rate shows how many times faster than normal the budget is burning. You build a multi-window alert: for example fast burn (burn rate around 14 over a 1 hour window, confirmed by a shorter 5 minute window) pages a person, slower burn (burn rate 6 over 6 hours) creates a ticket. The short window removes false alarms after the problem is already over. This way a night call means the budget is really disappearing.
Follow-up question: How long until the budget is exhausted at burn rate 14?
- Knowledge base: Prometheus, Grafana
- Source: sre.google
4. At 3 AM an alert fires: 30% of requests return 500, and you are on call. What are your first actions?
Answer
First I confirm the scale and user impact, announce it in the incident channel and take the incident owner role. The goal of the first minutes is mitigation, not finding the root cause. I look at what changed: a recent deploy, config, a dependency, traffic. If there is an obvious culprit, I roll back the release or switch off a feature flag. Then I narrow the cause using metrics, logs and traces. After stabilization I write a blameless postmortem: timeline, causes, actions so it does not repeat.
Follow-up question: When do you roll back a release and when do you fix forward?
5. Nobody knows what exactly runs in the prod cluster: people run
kubectl editandhelm upgradefrom their laptops, and the versions in Git stopped matching the cluster long ago. The team proposes Argo CD. What changes in who deploys and how, how rollbacks and secrets work, and what can go wrong?Answer
GitOps makes Git the single source of truth: an agent in the cluster (Argo CD, Flux) constantly compares the cluster with the repository and pulls changes itself, so a deploy is a merged PR and a rollback is a reverted commit. Write access to the cluster is taken away from people and CI: CI only builds the image and updates the tag in the repository, and the change audit becomes Git history. Manual edits are either reverted by the agent (self heal) or shown as drift, so an emergency
kubectl editmust be mirrored in Git right away. Secrets cannot go into Git in plain text; you need Sealed Secrets, SOPS or External Secrets. Risks: an agent with broad rights becomes a valuable target, a mistake in the repository spreads to the cluster instantly, and the team has to get used to 'just fix it by hand' no longer working during an incident.Follow-up question: How do you promote a version from staging to prod in a GitOps repository?
- Course: Kubernetes: the basics
- Knowledge base: Argo CD, Helm, Kubernetes
- Source: argo-cd.readthedocs.io
6. The incident is closed. How do you write a postmortem and what should it contain?
Answer
A blameless format: focus on the system and processes, not on the person who pressed the button. Parts: a short summary and user impact, a timeline with detection and mitigation times, the root cause and contributing factors, what went well, and concrete actions with owners and deadlines. Good actions reduce the chance or impact of a repeat rather than amounting to 'be more careful'. The document is shared with the team and action completion is tracked.
Follow-up question: How do you tell the root cause from contributing factors?
Train these questions
Study the cards, answer by text or voice and get an AI grade. Needs a signed in account.
Answer out loud and get graded
Here you get the questions with model answers. In the ShortLesson app you answer yourself, by text or voice, and AI grades the answer against the rubric and asks a follow-up, like a real interviewer.
The app is coming to the stores soon.
Other topics
- JavaScript interview questions and answers
- TypeScript interview questions and answers
- React interview questions and answers
- Frontend performance interview questions and answers
- Node.js interview questions and answers
- NestJS interview questions and answers
- HTTP interview questions and answers
- SQL interview questions and answers
- Databases interview questions and answers
- Git interview questions and answers
- Algorithms interview questions and answers
- System design interview questions and answers
- Web security interview questions and answers
- Testing interview questions and answers