Operations

Deployments

• Build from services/datalake/Dockerfile.
• The container runs granian against services.datalake.main:app.
• Default listen port is 8080 and honors the PORT environment variable.

Startup behavior

• Loads DATALAKE_SENTRY_DSN from Secret Manager.
• Initializes BigQuery client and Pub/Sub publisher/subscriber.
• Starts a Pub/Sub subscription callback for change stream messages.

Monitoring and alerts

• Sentry is required; startup fails if the DSN is missing.
• Health endpoint exposes start_time and messages_processed.

SLOs and SLIs

• TBD. Define throughput and error targets for change stream ingestion.

Scaling and performance

• Stateless service with Pub/Sub flow control.
• Ack deadlines are extended while processing large payloads.
• BigQuery inserts retry up to three times per message.