Checklists

Short lists to run through so that nothing obvious is missed. Copy them into your test plan or your pull request template, and delete the lines that do not apply.

One endpoint

Works

[ ] A valid request returns the right status (the exact code, not any 2xx)
[ ] The body has the right values
[ ] The body has every promised field, with the right type
[ ] Content-Type matches the body
[ ] It answers within the agreed time

Refuses

[ ] Each required field, when missing, gives 400 or 422
[ ] A wrong type gives 400 or 422
[ ] A value just outside each limit is refused; one just inside is accepted
[ ] An id that does not exist gives 404
[ ] Broken JSON gives 400, not 500
[ ] The wrong method gives 405
[ ] No bad input produces a 500

Is locked

[ ] No credentials gives 401
[ ] Bad or expired credentials give 401
[ ] A user without permission gets 403
[ ] One user cannot read or change another user's data

Leaves things right

[ ] After a create, a read returns the new item
[ ] After an update, a read returns the new values
[ ] After a delete, a read gives 404
[ ] A refused request changed nothing
[ ] Sending the same request twice does what the design says

Says no helpfully

[ ] Errors have a consistent shape across the API
[ ] The message says what to fix
[ ] No stack trace, query or internal path leaks out

A whole API

[ ] Every endpoint has at least one passing happy-path test
[ ] Every endpoint that takes input has negative tests
[ ] Every protected endpoint has a no-credentials test
[ ] The main user journeys run end to end as workflows
[ ] Lists handle empty results, paging and filters
[ ] The address and credentials are variables, not written into requests
[ ] The suite passes three times in a row
[ ] The suite runs in CI
[ ] The critical journey has a load baseline on record

Before a release

[ ] The full regression suite is green against the release candidate
[ ] Tests exist for everything new in this release
[ ] Tests exist for every bug fixed in this release
[ ] Nothing in the contract changed without the clients knowing: no removed field, renamed field or changed status
[ ] Response times are no worse than the last release's baseline
[ ] Known failures are written down, with an owner
[ ] The monitor is updated for any new critical endpoint

A good bug report

[ ] A title that states the problem: POST /orders accepts a quantity of 0
[ ] The environment and version
[ ] The exact request: method, URL, headers and body. Copying the request as cURL from ApiWorkbench captures all of it
[ ] What you expected, and where that expectation comes from
[ ] What happened: status, body and time
[ ] Whether it happens every time
[ ] Secrets removed. Check tokens in headers and bodies before pasting

See cURL in and out for copying a request as a command.

Reviewing someone's tests

[ ] Could each test fail? What would have to break?
[ ] Is anything checked beyond the status?
[ ] Is there a negative test for each rule?
[ ] Are the names clear enough to read a failure without opening the request?
[ ] Does any test depend on data that might not be there?
[ ] Does any test depend on another's leftovers, without that being the point?
[ ] Are there secrets in the file?
[ ] Would a newcomer know how to run it?

A mock, before you rely on it

[ ] Its paths, fields and status codes match the agreed contract
[ ] It has error responses, not only successes
[ ] Protected endpoints require the auth header
[ ] It has no real personal data or real secrets
[ ] The project file is committed and reviewed
[ ] Someone will update it when the real API changes

Checklists ​

One endpoint ​

A whole API ​

Before a release ​

A good bug report ​

Reviewing someone's tests ​

A mock, before you rely on it ​

Next steps ​