Watcher
Overview
Python 3.13 pipeline stall detector
Watcher is a Python 3.13 Cloud Run Job that checks Spanner pipeline metadata and logs warnings when pipelines appear stalled.
Job profile
| Field | Value |
|---|---|
| Code | jobs/watcher/ |
| Package | watcher |
| Runtime | Python 3.13 (Cloud Run Job) |
| Status | Active |
| Primary owner | Joe Pardi |
| Secondary owner | None |
| Trigger | Cloud Run Job (manual or scheduler) |
| Data source | Cloud Spanner (Pipelines) |
| Output | Logs + Sentry |
Purpose
- Detect pipelines that have not updated within their expected delay windows.
- Provide lightweight monitoring for potential ingestion stalls.
Non-goals
- Scheduling or repairing pipelines.
- Publishing metrics beyond logs and Sentry.
Inputs and outputs
- Inputs: Spanner
PipelinesandOrganizations. - Outputs: Logs and Sentry errors for stalled pipelines.
Related pages
Last updated on