Runbooks

Pub/Sub backlog growth

Symptoms

• Subscription backlog increases and messages_processed stalls.

Checks

1. Verify the subscription name and IAM permissions.
2. Inspect Cloud Run CPU/memory usage and scale limits.
3. Reduce MAX_MESSAGES or MAX_BYTES if memory pressure occurs.

BigQuery insert errors

Symptoms

• Errors inserting into BigQuery; messages are nacked.

Checks

1. Confirm dataset and <tableName>_changelog tables exist.
2. Validate BigQuery permissions for the service account.
3. Check schema compatibility with the changelog format.

Missing downstream topics

Symptoms

• Downstream consumers do not receive Pub/Sub updates.

Checks

1. Verify topic creation permissions in the BigQuery project.
2. Inspect logs for topic creation errors.

Health checks

• GET /api/v1/health returns status and processing metrics.