AW Dev Rethought

βœ‚οΈ Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away - Antoine de Saint-ExupΓ©ry

πŸ”¬ Python Code Lab β€” Does functools.cache Actually Make It Faster? 🐍


Description:

Caching helps when a later call can reuse an earlier result. To measure that benefit, we need to distinguish an empty cache, a prefilled cache, and inputs that never repeat.


Keep the work identical

Our deliberately expensive, deterministic function performs 200,000 iterations. It returns an immutable integer; there is no sleep, file access or network call.

from functools import cache

def expensive(n):
    return sum((i * i) % 97 for i in range(n))

cached = cache(expensive)

cache(expensive) wraps the same implementation. It is equivalent to applying @cache at definition time. Calling either version with 200,000 returns 9,600,239.


Count misses before interpreting timings

For 20 identical inputs and an initially empty cache:

cached.cache_clear()
for _ in range(20):
    cached(200_000)
print(cached.cache_info())
# hits=19, misses=1, maxsize=None, currsize=1

The first call computes and stores the answer. The next 19 calls retrieve it. Another batch of the same inputs adds 20 hits without another miss. With 20 distinct arguments and an empty cache, there are 20 misses and 20 entries.


Separate cold and warm measurements

A timed batch builds a list of 20 return values. Timer.setup clears or prefills the cache outside the timed interval.

from timeit import Timer

repeated = [200_000] * 20
cold = Timer(lambda: [cached(n) for n in repeated],
             setup=cached.cache_clear)
# One batch per cold trial: one miss, then nineteen hits.
cold_samples = cold.repeat(repeat=7, number=1)

For warm trials, we first clear and prefill one entry. Each trial then runs 10,000 batches; divide its elapsed time by 10,000 to report one 20-call batch. Prefilling is excluded, while the batch loop and list construction are included. We shuffle case order across seven trials and retain all samples. timeit temporarily disables cyclic garbage collection by default.


Measured results

CPython 3.11.9 on macOS ARM64. Minimum of seven trials; milliseconds per 20-call batch.

Workload Minimum ms Median ms
Uncached, repeated input 154.163 155.292
Cache starts empty, repeated input 7.589 7.832
Prefilled cache, repeated input 0.000736 0.000758
Uncached, distinct inputs 153.961 155.111
Cache starts empty, distinct inputs 154.437 155.015

The prefilled result is approximately 0.74 microseconds per batch, not milliseconds. It measures lookups plus loop/list overhead, excluding the computation that populated the cache. It is not an end-to-end first-request latency.

The distinct-input results are close. This run does not establish a meaningful speed advantage or penalty for that case. The important observation is that every unique input still requires computation.


When caching fits

Use it when expensive results can be reused for repeated, hashable arguments. cache has no size bound and retains arguments/results until cleared; a bounded lru_cache(maxsize=...) can limit entries. Neither provides automatic freshness for changing external data.

Side effects, mutable return values and changing inputs need care. A thread-safe cache also does not guarantee one execution per key when concurrent misses overlap. The notebook uses a single process and does not benchmark memory or concurrency.

Caching avoids repeated computation; it does not accelerate the function body. Measure the input distribution and cache lifecycle your application actually uses.


Link copied!

Comments

Add Your Comment

Comment Added!