Settings page regression tests (1/6): foundation and Features tab by conatus · Pull Request #133 · commonknowledge/join

conatus · 2026-10-23T19:06:41Z

First of six PRs pinning down what every setting on WP Admin > Join does today, so the settings page can be rebuilt without Carbon Fields and checked against them. The later PRs (Theme + Copy, Membership Plans, Integrations, Logging + Debug, then type and format checks for the test suites) stack on this one.

How the tests work

Each test changes a setting through the settings page, then checks:

It was saved: read back through the plugin's own Settings::get, not the raw rows, so the check survives a change of storage format.
It has its effect: on the join form's settings (<script id="env">), on what the form shows and does, and on what the plugin sends to other services.

Each tab also gets a legacy-data test: settings loaded straight into the database in today's Carbon Fields format (tests/settings/legacy/*.json), standing for existing sites, which must keep working after the rebuild.

Foundation (`packages/join-e2e/tests/settings`)

JoinSettingsPage (support.ts) is the only code that knows how the settings page is built. It finds fields by their visible labels. After the rebuild, update it (and any changed labels); the specs shouldn't need to change.
Isolation: every test starts from the seeded site plus one global membership plan, because the settings page refuses to save without one. seed.sh now snapshots the options table, and tests restore it afterwards, so the other specs are unaffected.
Fake third-party APIs (fake-api/): a small Node container standing in for Stripe, Mailchimp, Action Network, Zetkin, GoCardless, Chargebee, Auth0 and postcode lookups.
- Inside the Docker network their real hostnames resolve to it.
- It answers over HTTPS with a test-only certificate that only the E2E containers trust, so the plugin's real SDKs run unchanged and nothing reaches the real services.
- It records every request for tests to check. Stripe gets a tiny in-memory fake, so products and prices created by the plugin are found again.
- The committed key and CA are test fixtures; the CA's private key was deleted after signing, so they can't sign anything else.
Faster WP-CLI: the CLI container now stays running and commands are exec'd, about 0.5s instead of 1.5s per call. This also removes the --progress flag older Compose versions rejected.

Features tab (`features.spec.ts`, `features-integrations.spec.ts`)

Form settings: date of birth (and required); minimum age, default 18 and raised; "how did you hear about us" (with "Other" details); county (required); contact consent (optional, pre-ticked, email/phone required); international phone numbers; hiding £0 prices.
Integration switches: each of the seven is refused without its credentials, and then nothing is saved. Each is saved with them, and GoCardless Custom Pages requires GoCardless.
Payment-step effects:
- provider logos;
- GoCardless hosted pages (the redirect call and its payload) versus Custom Pages (bank details in the form);
- Chargebee set up with the site and publishable key (via a stand-in script);
- "How would you like to pay?" when two providers offer different methods;
- the Stripe publishable key reaching the form;
- the Auth0 password field and its strength rules.
Use Stripe on save: each global plan is set up in Stripe (product and recurring price, with the secret key), the IDs are stored against the plan, and saving again reuses them.
Legacy data: every Features setting in today's storage format still works.

Found along the way

JOIN-161: with Use Stripe on, a key Stripe rejects (or Stripe being unreachable) makes saving the settings page end in a PHP fatal error page. test.fixme until fixed.
Behaviour the rebuild must keep: saving drops values of hidden conditional fields. For example, "Direct Debit only" is cleared unless "Enable direct debit through Stripe" is ticked. This will be pinned in the Integrations PR.
Browser-only checks: age and consent rules are enforced in the browser only; the server never checks them. Noted, not changed.

Results

The full E2E suite passes locally in a cloud session: 122 passed, 2 skipped (JOIN-158, JOIN-161), in 7.3 minutes, previously about 2.5 minutes.

Generated by Claude Code

Tests that pin down what every setting on WP Admin > Join does today, so the settings page can be rebuilt without Carbon Fields and checked against them. Each test changes a setting through the settings page, checks it was saved (read back through Settings::get, so the check survives a storage change) and that it has its effect on the join form and on what the plugin sends out. Foundation (tests/settings): - JoinSettingsPage: the only code that knows how the settings page is built; finds fields by their visible labels. - Each test starts from the seeded site plus one global plan (the settings page refuses to save without one); seed.sh snapshots the options table and tests restore it afterwards. - Legacy fixtures: settings as Carbon Fields stores them, loaded straight into the database, to check existing sites keep working after the rebuild. - A fake-API container standing in for Stripe, Mailchimp, Action Network, Zetkin and the other services: their hostnames resolve to it inside the Docker network and it answers over HTTPS with a test certificate only the E2E containers trust, so the plugin's real SDKs run unchanged. It records requests for tests to check. - WP-CLI now runs in a long-lived container (exec, not run), cutting each call from ~1.5s to ~0.5s. Features tab: every setting, the integration switches' credential checks, their effects on the payment step, and Use Stripe setting global plans up in Stripe on save. A rejected Stripe key crashes the save (Linear JOIN-161): test.fixme. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL2bFrTk84JMy3CPAsCTV9

The test CA was bind-mounted over wp-includes/certificates/ca-bundle.crt. On a fresh volume the WordPress image copies its files into place on first start, and tar can't replace the read-only mounted file, so the container exited (CI always starts fresh; locally the volume already existed). Point WordPress's HTTP API at the test CA with a must-use plugin instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL2bFrTk84JMy3CPAsCTV9

conatus

Supplementary review: test-assertion strength in the shared join-form helpers, with suggested fixes. These add to the inline findings already posted.

Generated by Claude Code

- Fake API: a malformed control request or override pattern gets a 400 instead of crashing the fake for the rest of the run; patterns are compiled when added. - Trust the test CA in the Chargebee SDK as well as Stripe's. - scripts/env.sh wp: say to run `scripts/env.sh start` when the containers aren't running. - Keep storage knowledge in one place: setBaselinePlan() (next to JoinSettingsPage) replaces resetToSettingsBaseline(), and the options table is restored once per test rather than twice. - One shared pattern for the join form's env script. - Remove the temp directory holding the admin's session after each worker. - expectDetailsRejected waits for the form to mark a field invalid; expectDetailsAccepted checks the details step is done. - The hide-zero-price test checks the free plan is still offered. - admin-plan-redirect.spec.ts uses the shared settings helpers, and its help-text test sets up its own plan instead of relying on the test before it. - Docs: no stale test count; legacy tests are per tab. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL2bFrTk84JMy3CPAsCTV9

linear-code · 2026-10-25T09:39:51Z

JOIN-173

claude added 2 commits October 23, 2026 19:06