Probes

Probes

Liveness, Readiness and Startup Probes

Each probe answers a different question and triggers a different action:

The three probes and what the kubelet does when each fails
Probe	Question	On failure
Startup	Has the application finished starting?	Holds off the other probes; restarts after its threshold
Liveness	Is the process stuck beyond repair?	The kubelet kills and restarts the container
Readiness	Can it serve requests right now?	The Pod leaves its Services' endpoints; no restart

Liveness is drastic, so it must test only what a restart can fix. BookNest's API was built for this split (API Pod Spec): /ready queries PostgreSQL 1,289 , /health does not. Take the database away:

Taking PostgreSQL away: readiness fails, liveness holdsShell

kubectl scale statefulset postgres --replicas=0 >/dev/null && sleep 25
kubectl get pods -l tier=api
IP=$(kubectl get gateway booknest -o jsonpath='{.status.addresses[0].value}')
curl -s -o /dev/null -w 'HTTP %{http_code}\n' --resolve books.example.com:80:$IP \
  http://books.example.com/api/books/3
kubectl scale statefulset postgres --replicas=1 >/dev/null
kubectl wait --for=condition=Ready pod -l tier=api --timeout=90s >/dev/null

Output

NAME                   READY   STATUS    RESTARTS   AGE
api-776c666c56-28p22   0/1     Running   0          41s
api-776c666c56-7wklm   0/1     Running   0          41s
api-776c666c56-p8thf   0/1     Running   0          41s
HTTP 503

The replicas went 0/1 without a restart, the Gateway answered 503, and all recovered once postgres-0 returned. Had liveness used /ready, the kubelet would have restarted every API container in a loop during the outage.

MENU