Integrations Smoke Test Runbook

Manual end-to-end test for OAuth + ingestion + proposal generation per provider. Run against Vercel Preview before promoting env vars to Production.

Status: HubSpot (Phase 2) and Google Drive (Phase 4) sections complete. QuickBooks / Gmail sections will be added as their phases land.

HubSpot

1. Prerequisites

  • HubSpot developer portal account with an app configured at https://developers.hubspot.com.
  • Redirect URI in the HubSpot app set to ${NEXT_PUBLIC_APP_URL}/api/integrations/hubspot/callback for the environment under test (Preview first).
  • App scopes configured (must match lib/integrations/providers.ts):
    • crm.objects.contacts.read
    • crm.objects.companies.read
    • crm.objects.deals.read
  • Vercel env vars (Preview, then Production after smoke pass):
    • HUBSPOT_CLIENT_ID
    • HUBSPOT_CLIENT_SECRET
    • INTEGRATIONS_HUBSPOT_ENABLED=true
  • Fly.io worker secrets (fly secrets list -a boots-proposal-worker):
    • HUBSPOT_CLIENT_ID
    • HUBSPOT_CLIENT_SECRET
    • BOOTS_BRAIN_URL
    • BRAIN_API_SECRET
    • INTEGRATIONS_USE_FIXTURES is unset or false (default).
  • Worker deployed and healthy: fly status -a boots-proposal-worker and fly logs -a boots-proposal-worker.

2. OAuth connect

  1. Sign in to the app under test as an org owner or admin (Connect requires admin/owner per PR #223).
  2. Navigate to /settings/integrations.
  3. Click Connect on the HubSpot card.
  4. Authorize in the HubSpot consent screen.
  5. Confirm redirect lands on /settings/integrations with a success state and the card now shows a Disconnect button.

Verification SQL (run in Supabase SQL editor against the env under test):

SELECT id, provider, status, last_synced_at, created_at
FROM connected_sources
WHERE org_id = '<your-org-id>'
  AND provider = 'hubspot'
  AND deleted_at IS NULL;
-- Expected: one row, status = 'active' (or 'pending' until first sync completes)

3. First sync

The OAuth callback enqueues a full_sync job on pgmq. The Fly.io worker should pick it up within INGESTION_POLL_INTERVAL_MS (default 5s).

Tail the worker:

fly logs -a boots-proposal-worker

Expected log lines: Token needs refresh (only if access token already expired), then job state transitions to running → completed.

Verification SQL:

SELECT id, status, items_total, items_succeeded, items_failed, error_message, completed_at
FROM ingestion_jobs
WHERE org_id = '<your-org-id>'
ORDER BY created_at DESC
LIMIT 3;
-- Expected: status = 'completed', items_failed = 0

SELECT document_type, count(*)
FROM source_documents
WHERE org_id = '<your-org-id>'
GROUP BY document_type;
-- Expected: rows for hubspot.contact, hubspot.company, hubspot.deal

SELECT client_name, source_id, created_at
FROM client_intelligence
WHERE org_id = '<your-org-id>'
  AND source_id IS NOT NULL
LIMIT 10;
-- Expected: one row per HubSpot company with source_id populated

4. Proposal generation

Create a new proposal in the app for a client whose name matches a HubSpot company name from step 3. After generation completes, query attribution:

SELECT sa.agent_role, sa.section_name, ci.client_name, cs.provider
FROM source_attribution sa
JOIN client_intelligence ci ON ci.id = sa.client_intelligence_id
JOIN connected_sources cs ON cs.id = ci.source_id
WHERE sa.proposal_id = '<proposal-id>';
-- Expected (after Step 9 / Phase 3 ships): rows showing HubSpot data was attributed
-- Note: pre-Step-9, this query returns 0 rows even on a successful proposal

5. Token refresh

  1. Note the current token_expires_at for the HubSpot row in connected_sources.
  2. Manually backdate it via Supabase SQL: 
    UPDATE connected_sources
SET token_expires_at = '2000-01-01T00:00:00Z'
WHERE org_id = '<your-org-id>'
  AND provider = 'hubspot';
  3. Trigger a manual refresh (re-enqueue from the UI or insert a pgmq message manually).
  4. Watch worker logs for Token needs refresh followed by a successful sync.
  5. Verify new token_expires_at is in the future and no Sentry errors fired.

6. Error path — revoked token

  1. In HubSpot developer portal, revoke the connected app's access for your test account (User → Connected Apps → Disconnect).
  2. Trigger a manual sync.
  3. Expected:
    • connected_sources.status = 'error'
    • status_message contains refresh_token_invalid (see worker/src/integrations/token-refresh.ts:108-116).
    • /settings/integrations shows an error badge and a Disconnect button so the user can reconnect.

7. Failure modes — quick reference

Symptom in UI Root cause Fix
HubSpot card shows "Coming soon" INTEGRATIONS_HUBSPOT_ENABLED not 'true' in this env Set the env var in Vercel for the matching environment
Connect button greyed with "OAuth not configured" HUBSPOT_CLIENT_ID or HUBSPOT_CLIENT_SECRET missing in Vercel Set both in Vercel env, redeploy
Connect succeeds, status stays pending Worker hasn't picked up the job, or callback didn't enqueue fly logs -a boots-proposal-worker; check ingestion_jobs for the row
ingestion_jobs.status='failed' with 403 in error_message Missing crm.objects.companies.read scope on HubSpot app Add scope in HubSpot app, disconnect, reconnect
ingestion_jobs.status='failed' with brain URL error Worker missing BOOTS_BRAIN_URL / BRAIN_API_SECRET fly secrets set both, restart worker
source_documents empty after sync Real client returned 0 results, or wrong scopes Confirm test HubSpot account actually has contacts/companies/deals
client_intelligence empty but documents exist Companies have no name property on payload Check raw payloads in source_documents.payload
connected_sources.status='error' after refresh refresh_token rejected (user revoked or HubSpot rotated) User must Disconnect → Connect again

8. After smoke passes

Promote env vars to Production:

INTEGRATIONS_HUBSPOT_ENABLED=true
HUBSPOT_CLIENT_ID=…
HUBSPOT_CLIENT_SECRET=…

Add the production redirect URI to the HubSpot app. Repeat steps 2–6 against Production with a real customer account once.

Google Drive

1. Prerequisites

  • Google Cloud project with an OAuth 2.0 Web client at https://console.cloud.google.com/apis/credentials.
  • Authorized redirect URI on the OAuth client set to ${NEXT_PUBLIC_APP_URL}/api/integrations/google_drive/callback for the environment under test (Preview first).
  • OAuth consent screen scopes (must match lib/integrations/providers.ts):
    • https://www.googleapis.com/auth/drive.readonly
  • Drive API enabled in the same Google Cloud project (APIs & Services → Library → Google Drive API → Enable).
  • Vercel env vars (Preview, then Production after smoke pass):
    • GOOGLE_CLIENT_ID
    • GOOGLE_CLIENT_SECRET
    • INTEGRATIONS_GOOGLE_DRIVE_ENABLED=true
  • Fly.io worker secrets (fly secrets list -a boots-proposal-worker):
    • GOOGLE_CLIENT_ID
    • GOOGLE_CLIENT_SECRET
    • BOOTS_BRAIN_URL
    • BRAIN_API_SECRET
    • INTEGRATIONS_USE_FIXTURES is unset or false (default).
    • Optional: DRIVE_INITIAL_SYNC_MAX_FILES (default 200).
  • Worker deployed and healthy: fly status -a boots-proposal-worker.
  • Test Google account has at least a few Google Docs and Sheets in its Drive (other file types are filtered out by the client).

2. OAuth connect

  1. Sign in to the app under test as an org owner or admin (Connect requires admin/owner).
  2. Navigate to /settings/integrations.
  3. Click Connect on the Google Drive card.
  4. Authorize in the Google consent screen — make sure the Drive read-only scope is granted.
  5. Confirm redirect lands on /settings/integrations with the card now showing a Disconnect button.

Verification SQL:

SELECT id, provider, status, last_synced_at, created_at
FROM connected_sources
WHERE org_id = '<your-org-id>'
  AND provider = 'google_drive'
  AND deleted_at IS NULL;
-- Expected: one row, status = 'active' (or 'pending' until first sync completes)

3. First sync

The OAuth callback enqueues a full_sync job on pgmq. The Fly.io worker picks it up within INGESTION_POLL_INTERVAL_MS (default 5s). Initial sync is capped at DRIVE_INITIAL_SYNC_MAX_FILES (default 200) Docs/Sheets.

fly logs -a boots-proposal-worker

Verification SQL:

SELECT id, status, items_total, items_succeeded, items_failed, error_message, completed_at
FROM ingestion_jobs
WHERE org_id = '<your-org-id>'
ORDER BY created_at DESC
LIMIT 3;
-- Expected: status = 'completed', items_failed = 0

SELECT document_type, count(*)
FROM source_documents
WHERE org_id = '<your-org-id>'
GROUP BY document_type;
-- Expected: rows for google_drive.doc and/or google_drive.sheet

SELECT external_id, raw_payload->>'name' AS name,
       raw_payload->>'mimeType' AS mime,
       length(raw_payload->>'exported_text') AS exported_chars
FROM source_documents
WHERE org_id = '<your-org-id>'
  AND document_type LIKE 'google_drive.%'
ORDER BY exported_chars DESC
LIMIT 10;
-- Expected: file names match what's in your Drive,
-- exported_chars > 0 for non-empty Docs/Sheets.

Note: Google Drive does not produce client_intelligence rollups (see worker/src/handlers/ingestion.ts — buildRollups returns [] for google_drive). Drive content reaches the strategist via source_documents semantic search, not rollups.

4. Proposal generation

Drive documents are surfaced to the strategist through source_documents semantic search rather than client_intelligence rollups. To verify Drive content can influence a proposal:

  1. Ensure the test Drive contains a Doc whose body mentions a known client name (e.g., the client you'll generate for).
  2. Generate a proposal for that client.
  3. Inspect the strategist context (or check Sentry / worker logs) to confirm the Doc was retrieved.

Source attribution for vector-search hits is not wired yet — see the match_client_intelligence follow-up in docs/plans/PRD-integrations-steps8to10.md.

5. Token refresh

  1. Note the current token_expires_at for the Drive row in connected_sources.
  2. Backdate it via Supabase SQL: 
    UPDATE connected_sources
SET token_expires_at = '2000-01-01T00:00:00Z'
WHERE org_id = '<your-org-id>'
  AND provider = 'google_drive';
  3. Trigger a manual refresh (re-enqueue from the UI or insert a pgmq message manually).
  4. Watch worker logs for Token needs refresh followed by a successful sync.
  5. Verify new token_expires_at is in the future and no Sentry errors fired.

Google refresh tokens require access_type=offline and prompt=consent on the auth request — these are already set in PROVIDERS.google_drive.extraAuthParams (see lib/integrations/providers.ts). If connected_sources.refresh_token_vault_id is null after the first connect, the consent screen ran without prompt=consent — disconnect and reconnect.

6. Error path — revoked token

  1. Revoke the connected app's access at https://myaccount.google.com/permissions (find the test app and remove access).
  2. Trigger a manual sync.
  3. Expected:
    • connected_sources.status = 'error'
    • status_message contains refresh_token_invalid (see worker/src/integrations/token-refresh.ts).
    • /settings/integrations shows an error badge and a Disconnect button so the user can reconnect.

7. Failure modes — quick reference

Symptom in UI Root cause Fix
Drive card shows "Coming soon" INTEGRATIONS_GOOGLE_DRIVE_ENABLED not 'true' in this env Set the env var in Vercel for the matching environment
Connect button greyed with "OAuth not configured" GOOGLE_CLIENT_ID or GOOGLE_CLIENT_SECRET missing in Vercel Set both in Vercel env, redeploy
Connect succeeds, status stays pending Worker hasn't picked up the job, or callback didn't enqueue fly logs -a boots-proposal-worker; check ingestion_jobs for the row
ingestion_jobs.status='failed' with 403 in error_message Drive API not enabled in Google Cloud project, or scope missing Enable Drive API; verify drive.readonly granted; reconnect
ingestion_jobs.status='failed' with 429 after retries Drive rate limit exceeded; sync cap likely needs lowering Lower DRIVE_INITIAL_SYNC_MAX_FILES, retry the job
ingestion_jobs completes but items_total=0 No Google Docs or Sheets in the test account (binary files are filtered) Create a few Docs/Sheets and trigger a manual sync
source_documents.exported_text is empty Drive export endpoint returned empty body — likely a Doc with no content Open the Doc in Drive and confirm it has body text
connected_sources.status='error' after refresh Refresh token rejected (user revoked or Google rotated) User must Disconnect → Connect again with prompt=consent

8. After smoke passes

Promote env vars to Production:

INTEGRATIONS_GOOGLE_DRIVE_ENABLED=true
GOOGLE_CLIENT_ID=…
GOOGLE_CLIENT_SECRET=…

Add the production redirect URI to the Google Cloud OAuth client. Repeat steps 2–6 against Production with a real customer account once.

QuickBooks

Not yet implemented — see Phase 5.

Gmail

Not yet implemented — see Phase 6.

← PreviousIntegrations — Local Dev Testing GuideNext →The Problem We Solve

Ready to create AI-powered proposals?

Start Free