Activity & test runs

Activity is the first page after you sign in. It shows what your test systems actually did:

• API calls to mocks, and the webhooks they sent;
• captured emails, texts, phone calls, WhatsApp messages and push notifications;
• requests to request bins;
• mock IdP sign-ins.

Everything sits in one timeline, newest first. When your tests tag what they send with a run ID, Activity groups those events into test runs and tells you how each one went.

#The timeline

Filter by time range (15 minutes up to 7 days), by system, by status and by test run. The filters are in the page address, so you can share a view.

Each event has a status:

Select an event to see its details:

• API calls: the request and response, with credentials masked.
• Bin requests: the captured request.
• Everything else: a link to the system's own page.

The Systems tab gives each system a card with:

• its activity in the range;
• its configuration in brief (mock outcome and failure settings, bin answers and rules, SMS failure rules, and so on);
• the records an API mock stores;
• the failure scenarios that are on.

#Test runs

Add the header X-Testlie-Run: <run id> to the requests your tests make. A run ID is 1 to 100 letters, digits or . _ : / @ -, for example your CI run: X-Testlie-Run: gh/$GITHUB_RUN_ID/$GITHUB_JOB.

The header is recorded on:

• calls to API mocks (/mock/<slug>/…);
• requests to request bin capture URLs;
• the Twilio-compatible SMS, Verify and Voice APIs, the WhatsApp API, and the APNs and FCM push APIs;
• emails, as a message header X-Testlie-Run, whether they are sent over SMTP or arrive at an inbox address;
• the REST wait endpoints (…/messages/wait, …/requests/wait, …/notifications/wait, …/calls/wait), which your tests use to wait for results.

Messages that arrive over SMPP and mock IdP sign-ins carry no header and show up untagged.

#Outcome

A run's outcome is worked out in this order:

1. Failed if the test reported a failure, or if any wait in the run timed out, because a wait is an expectation that didn't happen.
2. Passed if the test reported a pass.
3. Errors seen if nothing was reported but the run has error events.
4. Passed if all of the run's waits matched.
5. No result otherwise. You still see what the run did.

Report your test's own result when it finishes:

curl -X POST "$TESTLIE_URL/api/runs/$RUN" -H "Authorization: Bearer $TESTLIE_API_KEY" \
  -H "Content-Type: application/json" -d '{"status": "passed", "summary": "42 tests"}'

The latest report wins. A wait that timed out still fails the run.

#With the SDK and CLI

const testlie = new Testlie({ run: process.env.GITHUB_RUN_ID }); // or set TESTLIE_RUN
await fetch(`${stripeMock}/v1/customers`, { method: "POST", headers: { ...testlie.runHeaders, Authorization: `Bearer ${key}` }, body });
const mail = await testlie.waitForEmail({ inboxId, to }); // counted as a matched wait
await testlie.runs.report("passed", { summary: "checkout suite" });

export TESTLIE_RUN="gh/$GITHUB_RUN_ID"
npx testlie wait --inbox "$INBOX" --to user@example.com   # counted in the run
npx testlie run report --status passed
npx testlie run show                                      # exits 1 when the run failed

#API

Any team member or API key can use these. Reporting needs an owner, an admin or a write key.

Events are one-line summaries without secrets. Details come from each system's own endpoints, such as GET /api/mocks/:id/calls/:callId, where credentials are masked.