Record modes¶
The record mode controls what happens when a request is made under an active cassette.
| Mode | Behavior |
|---|---|
none |
Replay only. Raises NoMatchError if no recorded interaction matches. |
once |
Record if the cassette doesn't exist. Replay if it does. |
new_episodes |
Replay existing interactions. Record new ones. |
all |
Record everything, overwriting the cassette. |
rewrite |
Delete the cassette, then record everything. |
verify |
Send matched requests live. Raise ResponseMismatchError if the response differs from the recording. |
none¶
Use it in CI and for day to day test runs. Nothing touches the network. If a request has no matching interaction in the cassette, the test fails with NoMatchError.
once¶
Use it while developing. The first run records, every run after that replays.
This is the default for use_cassette().
new_episodes¶
Use it when you add new requests to an existing test. Matched requests replay from the cassette, unmatched requests go to the real server and get appended.
all¶
Use it to re-record a cassette from scratch, for example after an API change.
rewrite¶
Same as all, except the cassette file is deleted before the test runs. A test that no longer makes the request it used to leaves no cassette behind, instead of leaving an empty one.
verify¶
Use it when the cassette is the expected output of the service under test, not a stand-in for a dependency. For example, when you port an HTTP API to another language: record the reference implementation once, then check the new one against it.
import httpx2
from cassetter import Cassetter, Compare
cassetter = Cassetter(
cassette_library_dir="tests/expected",
match_on=["method", "uri", "body"],
compare=Compare(
headers=["content-type", "x-price-*"],
json_ignore=["id", "created"],
float_tolerance=1e-9,
),
)
# Against the reference implementation.
with cassetter.use_cassette("chat.yaml", record_mode="rewrite"):
httpx2.post("http://localhost:8000/v1/chat", json={"model": "m", "messages": []})
# Against the new implementation.
with cassetter.use_cassette("chat.yaml", record_mode="verify"):
httpx2.post("http://localhost:8000/v1/chat", json={"model": "m", "messages": []})
Matching works as in none: match_on picks the recording, and a request without one raises NoMatchError. The difference is what happens after a match. The request goes to the network, the response goes through the same hooks, decompression and scrubbing as a recording, and is compared to the recorded one. The cassette is never written.
Compare decides what counts as equivalent:
| Field | Compared |
|---|---|
| status | Always, exactly. |
headers |
Only the listed names, by value. Globs such as x-price-* are allowed. date, server and every other header are ignored. |
json_ignore |
JSON bodies are compared structurally, without these paths. Same syntax as ignore_json_paths: id, data.created, items[0].id. |
float_tolerance |
Largest difference between two JSON numbers that still counts as equal. |
sse |
Compare the body as the list of Server-Sent Events data: payloads, so chunk boundaries and comments do not matter. JSON payloads are parsed, and json_ignore applies to each. |
Any other body is compared as text or bytes.
On a difference, the test fails with a unified diff of both responses, as compared:
cassette tests/expected/chat.yaml, interaction 0 (POST http://localhost:8000/v1/chat)
--- recorded
+++ live
@@ -1,7 +1,7 @@
status: 200
@@ headers @@
content-type: application/json
-x-price-estimate: 0.0001USD
+x-price-estimate: 0.0002USD
@@ json body @@
{
"content": "hi"
Normalize what Compare cannot express
before_compare_response receives the recorded and the live HttpResponse and returns both, normalized. For example, to treat an absent key and a null value as equal:
from cassetter import Body, HttpResponse
def drop_nulls(recorded: HttpResponse, live: HttpResponse) -> tuple[HttpResponse, HttpResponse]:
def clean(response: HttpResponse) -> HttpResponse:
content = {key: value for key, value in response.body.content.items() if value is not None}
return response.replace(body=Body("json", content))
return clean(recorded), clean(live)
gRPC and WebSockets¶
verify covers gRPC calls and WebSocket connections too. A matched call or connection goes live, and what comes back is compared to the recording when the call or connection ends.
import grpc
from cassetter import use_cassette
with use_cassette("echo.yaml", record_mode="verify", intercept=["grpc", "websockets"]):
async with grpc.aio.insecure_channel("localhost:50051") as channel:
response = await EchoStub(channel).Say(StringValue(value="hi"))
| Protocol | Compared |
|---|---|
| gRPC | The status code and message, then each response message in order. Messages are compared as JSON when google.protobuf is available, so json_ignore and float_tolerance apply to each. |
| WebSockets | Every frame the client received, in order, including the close code and reason. Received frames go through before_record_ws_frame and scrubbing first, like a recording. Text frames that hold JSON are parsed, and json_ignore and float_tolerance apply to each. Sent frames are not compared. |
A live gRPC error is compared like a response. If the recording holds the same status, the error is raised as it would be without a cassette.
cassette tests/expected/echo.yaml, gRPC /pkg.Echo/Say
--- recorded
+++ live
@@ -1,2 +1,2 @@
status: 0 OK
-message 1: echo bye
+message 1: echo hi
Response metadata is not compared
Cassettes do not store gRPC response metadata, so there is nothing to compare it with. Compare.headers and before_compare_response apply to HTTP only.
Observed requests¶
The service under test often sends HTTP requests of its own that the client never sees, such as telemetry exports to a collector. cassetter cannot intercept them, but your test runs the receiver, so the receiver hands them in with observe():
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import httpx2
from cassetter import RawRequest, active_cassette, use_cassette
class Collector(BaseHTTPRequestHandler):
def do_POST(self) -> None:
body = self.rfile.read(int(self.headers["content-length"]))
headers = {name.lower(): [value] for name, value in self.headers.items()}
cassette = active_cassette()
if cassette is not None:
cassette.observe(RawRequest("POST", f"http://collector{self.path}", headers, body))
self.send_response(200)
self.send_header("content-length", "0")
self.end_headers()
collector = ThreadingHTTPServer(("127.0.0.1", 4318), Collector)
threading.Thread(target=collector.serve_forever, daemon=True).start()
with use_cassette("chat.yaml", record_mode="verify"):
httpx2.post("http://localhost:8000/v1/chat", json={"model": "m", "messages": []})
active_cassette() returns the cassette entered by use_cassette() or the pytest plugin, or None. It also works from the receiver's own thread, as long as a single cassette is active.
| Mode | What observe() does |
|---|---|
Recording (all, rewrite, new_episodes, once on a new cassette) |
Stores the request under observed_requests, after the interactions, scrubbed like them. |
verify |
Keeps the request. When the cassette exits, compares each one, in arrival order, to the recorded one. |
Replaying (none, once on an existing cassette) |
Nothing. |
The requests arrive on their own schedule, so on exit verify waits until as many arrived as were recorded, or until observe_timeout seconds (5 by default) pass. A missing, extra or different request raises ResponseMismatchError.
Flush before the cassette exits when recording
A recording cassette cannot know how many requests to wait for. Make the service flush its exports before the with block ends, or they are not recorded.
Run the receiver in another thread
The wait blocks the thread that exits the cassette. A receiver served from that same thread, or its event loop, never gets to run.
Observed requests have their own Compare, passed as observe_compare. The method, and the path and query of the URI are always compared. The host is not, since the receiver usually listens on a different port on every run.
Compare(decode=...) turns a body into a JSON-able value before comparing, for a format cassetter cannot read, such as OTLP protobuf. json_ignore and float_tolerance apply to what it returns:
from cassetter import Compare, use_cassette
from google.protobuf.json_format import MessageToDict
from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import ExportTraceServiceRequest
def decode_spans(body: bytes) -> object:
return MessageToDict(ExportTraceServiceRequest.FromString(body))
observe_compare = Compare(
headers=["content-type"],
json_ignore=["resourceSpans[0].scopeSpans[0].spans[0].traceId"],
decode=decode_spans,
)
with use_cassette("chat.yaml", record_mode="verify", observe_compare=observe_compare):
...
decode works the same way in compare, for response bodies.
Set the mode from the command line¶
With the pytest plugin, --record-mode overrides whatever the tests configure:
A common workflow:
- Locally, record new cassettes with
pytest --record-mode=once. - In CI, run plain
pytest. The plugin default isnone, so CI never makes real requests.