Nest Engineering Docs
Scrape

Local development

Run, debug, and test Scrape locally

Prerequisites

  • uv and Python 3.13.
  • Playwright browsers installed (python -m playwright install).
  • GCP credentials with access to Spanner and Secret Manager.
  • EzyVet credentials and proxy secrets available.

Required environment

export SPANNER_INSTANCE_ID="..."
export SPANNER_DATABASE_ID="..."

Optional environment

export GCP_PROJECT_ID="..."
export SENTRY_ENVIRONMENT="development"

Google Application Credentials

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"

Run locally

# From repo root
uv run --package scrape main.py

Tests

# From repo root
uv run --package scrape pytest -q

Debugging

  • Login failures usually indicate missing or invalid EzyVet secrets.
  • Proxy errors often point to missing EZYVET_PROXY_* secrets.
  • Spanner write errors usually point to schema drift or missing permissions.

Last updated on