π¬ Python Code Lab β Does functools.cache Actually Make It Faster? π
Posted on: November 10, 2026
Description:
Caching helps when a later call can reuse an earlier result. To measure that benefit, we need to distinguish an empty cache, a prefilled cache, and inputs that never repeat.
Keep the work identical
Our deliberately expensive, deterministic function performs 200,000 iterations. It returns an immutable integer; there is no sleep, file access or network call.
from functools import cache
def expensive(n):
return sum((i * i) % 97 for i in range(n))
cached = cache(expensive)
cache(expensive) wraps the same implementation. It is equivalent to applying @cache at definition time. Calling either version with 200,000 returns 9,600,239.
Count misses before interpreting timings
For 20 identical inputs and an initially empty cache:
cached.cache_clear()
for _ in range(20):
cached(200_000)
print(cached.cache_info())
# hits=19, misses=1, maxsize=None, currsize=1
The first call computes and stores the answer. The next 19 calls retrieve it. Another batch of the same inputs adds 20 hits without another miss. With 20 distinct arguments and an empty cache, there are 20 misses and 20 entries.
Separate cold and warm measurements
A timed batch builds a list of 20 return values. Timer.setup clears or prefills the cache outside the timed interval.
from timeit import Timer
repeated = [200_000] * 20
cold = Timer(lambda: [cached(n) for n in repeated],
setup=cached.cache_clear)
# One batch per cold trial: one miss, then nineteen hits.
cold_samples = cold.repeat(repeat=7, number=1)
For warm trials, we first clear and prefill one entry. Each trial then runs 10,000 batches; divide its elapsed time by 10,000 to report one 20-call batch. Prefilling is excluded, while the batch loop and list construction are included. We shuffle case order across seven trials and retain all samples. timeit temporarily disables cyclic garbage collection by default.
Measured results
CPython 3.11.9 on macOS ARM64. Minimum of seven trials; milliseconds per 20-call batch.
| Workload | Minimum ms | Median ms |
|---|---|---|
| Uncached, repeated input | 154.163 | 155.292 |
| Cache starts empty, repeated input | 7.589 | 7.832 |
| Prefilled cache, repeated input | 0.000736 | 0.000758 |
| Uncached, distinct inputs | 153.961 | 155.111 |
| Cache starts empty, distinct inputs | 154.437 | 155.015 |
The prefilled result is approximately 0.74 microseconds per batch, not milliseconds. It measures lookups plus loop/list overhead, excluding the computation that populated the cache. It is not an end-to-end first-request latency.
The distinct-input results are close. This run does not establish a meaningful speed advantage or penalty for that case. The important observation is that every unique input still requires computation.
When caching fits
Use it when expensive results can be reused for repeated, hashable arguments. cache has no size bound and retains arguments/results until cleared; a bounded lru_cache(maxsize=...) can limit entries. Neither provides automatic freshness for changing external data.
Side effects, mutable return values and changing inputs need care. A thread-safe cache also does not guarantee one execution per key when concurrent misses overlap. The notebook uses a single process and does not benchmark memory or concurrency.
Caching avoids repeated computation; it does not accelerate the function body. Measure the input distribution and cache lifecycle your application actually uses.
No comments yet. Be the first to comment!