Handler
Overview
Python 3.13 ETL handler for PIMS ingestion and Bubble sync
Handler is the primary Python 3.13 FastAPI ETL service. It ingests partner PIMS data, normalizes it, and loads it into Cloud Spanner. It also processes Bubble webhook events to keep Bubble data in sync with Spanner.
Service profile
| Field | Value |
|---|---|
| Code | services/handler/ |
| Package | handler |
| Runtime | Python 3.13 (FastAPI) |
| Status | Active |
| Primary owner | Joe Pardi |
| Secondary owner | Akansh Divker |
| Ingress | Internal only (Cloud Tasks) |
| Data sink | Cloud Spanner |
Responsibilities
- Execute ETL pipelines for PIMS integrations (ezyVet, Bitwerx, HAP).
- Normalize and upsert partner data into Spanner.
- Process Bubble webhook tasks and write Bubble entities to Spanner.
- Forward processed invoices back to Bubble via Cloud Tasks.
Non-goals
- Public API surface or direct partner traffic.
- Scheduling tasks (handled by Stream and Webhooks).
Tech stack
- Python 3.13 + FastAPI (ASGI).
- Granian server (HTTP/1.1) with ORJSONResponse for JSON.
- Polars for vectorized transform steps.
- Cloud Spanner async client + batch write utilities.
- Cloud Tasks for internal task ingress + invoice forwarding.
- Sentry (optional) for tracing and error reporting.
Code entrypoints
services/handler/main.py: FastAPI app, Sentry, Spanner lifecycle.services/handler/api/v1/endpoints/factory.py: dynamic pipeline endpoints.services/handler/pipelines/: ETL pipelines per vendor and webhooks.services/handler/core/extractors/: partner API extraction helpers.services/handler/core/loader/: Spanner upsert orchestration.services/handler/core/cloud_tasks.py: Bubble invoice task dispatch.
Dependencies
- Upstream: Cloud Tasks (scheduled by Stream/Webhooks).
- Downstream: Cloud Spanner; Bubble API (invoice forwarding).
- External: Partner Credentials service; Secret Manager; Sentry.
Related pages
Last updated on