Scrape
Local development
Run, debug, and test Scrape locally
Prerequisites
uvand Python 3.13.- Playwright browsers installed (
python -m playwright install). - GCP credentials with access to Spanner and Secret Manager.
- EzyVet credentials and proxy secrets available.
Required environment
export SPANNER_INSTANCE_ID="..."
export SPANNER_DATABASE_ID="..."Optional environment
export GCP_PROJECT_ID="..."
export SENTRY_ENVIRONMENT="development"Google Application Credentials
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"Run locally
# From repo root
uv run --package scrape main.pyTests
# From repo root
uv run --package scrape pytest -qDebugging
- Login failures usually indicate missing or invalid EzyVet secrets.
- Proxy errors often point to missing
EZYVET_PROXY_*secrets. - Spanner write errors usually point to schema drift or missing permissions.
Last updated on