Negative testing
A negative test sends something wrong on purpose and checks that the API says no in the right way. Users, other programs and attackers all send bad input sooner or later. What the API does then decides whether a mistake becomes a clear message or a corrupted record.
What "refuses properly" means
A good refusal has four properties. Test for each.
| Property | Check |
|---|---|
| The right status | 400 or 422 for bad input, 401 without credentials, 403 when not allowed, 404 for a missing thing. Never 200, never 500 |
| A useful body | A machine-readable code and a message a person can act on |
| Nothing leaked | No stack trace, SQL, file path or internal hostname in the body |
| Nothing changed | The bad request left no half-made record behind |
A 500 in answer to bad input is a bug by definition: the server crashed where it should have validated.
A catalogue of negative cases
Work through this for each endpoint. Not every row applies to every endpoint.
The body
| Send | Expect |
|---|---|
| A required field left out | 400 or 422, naming the field |
A required field set to null or "" | 400 or 422 |
The wrong type: "quantity": "two" | 400 or 422 |
| A value outside the allowed range | 400 or 422 |
A value not in the allowed list: "status": "flying" | 400 or 422 |
| A very long string | 400, 413 or 422. Not a crash |
| Broken JSON: a missing brace | 400 |
| An empty body where one is required | 400 |
The wrong Content-Type | 400 or 415 |
The URL
| Send | Expect |
|---|---|
| An id that does not exist | 404 |
An id of the wrong form: /orders/abc | 400 or 404 |
| A path that does not exist | 404 |
| A method the endpoint does not support | 405 |
An invalid query value: ?page=-1 | 400, or a sensible default. Decide which |
Identity
| Send | Expect |
|---|---|
| No credentials | 401 |
| Wrong or expired credentials | 401 |
| Valid credentials, another user's resource | 403 or 404 |
These have their own guide: Testing authentication.
Build one in ApiWorkbench
Using POST /todos from the Todo API sample, which needs a title.
Duplicate the working Create a todo request, and rename the copy
Create a todo without a title.In Body, replace the JSON with
{}.In Tests, set Status equals to
400.Add JSON field equals: path
error, valuevalidation_failed.Select Send. Both pass: the API refused, and said why.
Duplicating the happy-path request and breaking one thing is the reliable way to write negative tests. If you break two things, you do not know which one the API objected to.
In the runner, a 4xx with tests is a pass
The runner treats a request with no tests as failed when the status is 400 or above. Give every negative request at least a Status equals test, so that the expected 400 counts as a pass. See Collection runner.
Many bad inputs, one request
Ten variations of a bad body do not need ten requests. Put the inputs and the expected status in a data file and run one request against it:
quantity,expectedStatus
0,422
-1,422
11,422
abc,400See Data-driven testing.
Negative tests against a mock
A mock only refuses what you taught it to refuse. In Mock Studio, give the endpoint Conditional rules: a response for the valid case with its conditions, and a fallback with the 400. The Shop API's Create order does this for a missing field and for a quantity over 10. See Conditional rules.
That makes a mock a good place to write negative tests and to check that a client shows errors properly. The proof that the real API validates comes from running the same tests against the real API.
Reading an error body
A good error is consistent across the API. Check that every negative test sees the same shape:
{
"error": "validation_failed",
"message": "title is required"
}If one endpoint answers with error and another with errorCode, clients have to handle both. That inconsistency is a finding worth reporting.
Common mistakes
- Accepting any 4xx. 400 and 404 mean different things. Test for the one that is right.
- Not checking the body. A 400 for the wrong reason passes a status-only test.
- Stopping at the first field. Test each required field's absence separately.
- Forgetting the aftermath. After a refused create, list the items and confirm nothing was added.