Nest Engineering Docs
Scrape

Inputs and outputs

Data contracts for Scrape

Inputs

  • Spanner: Clinics and Organizations tables for scrape configuration.
  • EzyVet UI: CSV reports downloaded via Playwright:
    • Agenda
    • Animal
    • Contact
    • Financial
    • Receipts
  • Secrets: EzyVet login credentials and proxy credentials.

Outputs

  • Cloud Spanner tables:
    • Households
    • Contacts
    • Patients
    • Appointments
    • Invoices
    • InvoiceLines
    • TeamMembers
  • Optional: MigrationMember (currently unused).

Contracts

  • CSV schema changes must be reflected in the transform helpers.
  • Spanner schema changes must remain backward compatible.

Last updated on

Architecture

Pipeline design and dependencies for Scrape

Schedule and triggers

Scheduling, triggers, and concurrency for Scrape