∇Continuous Function

Transformer Systems Lab

Long context station

Carry the KV-cache invariant into the harder question: a longer window can store more tokens, but that is not yet evidence that the model can reliably use them.

station 06local LabCase v1prediction firstserving handoff
Start Long ContextBack to KV MemoryOpen Route Cockpit
Selected objectLong-context constraint stationavailable tokens != reliable use
MemoryT * H_kvstill grows
Retrievalmiddle spanharder to use
ServingTTFT / TPOTnext pressure
01Object

evidence inside a longer window

02Gap

available tokens vs usable context

03Prediction

choose the pressure first

04Manipulation

T, position, task, distractors

05Evidence

position-sensitive toy witness

06Invariant

fitting tokens is not sufficient

07Next

serving workload pressure

Carry-in contract

Memory availability is not the same as reliable use.

KV memory answered a storage question. Long context turns it into an evaluation and systems question: where does retrieval fail, what positional range is being trusted, and what does serving pay?

Carried objectKV cache invariant: memory scales with T * H_kv.
PredictionWhich pressure must be tested before believing a longer window?
EvidencePosition-sensitive toy retrieval under task and distractor changes.
Next moveServing and decoding: turn the same long prompt into workload pressure.

Operable Station 06

Long-context use

Checking local routeNo prediction yet
  1. 01Text to one update
  2. 02Tokens and position
  3. 03Attention routing
  4. 04RoPE phase
  5. 05KV memory
  6. 06Long-context pressure
  7. 07Serving and decoding
  8. 08Speculative decoding
  9. 09Evaluation and falsification
  10. 10Capstone systems claim

Carry-in from KV memory

Reduced cache width is history; usable context is the new object.

The prior station saved the invariant that KV memory scales with T * H_kv. This station asks what remains uncertain when T grows: retrieval, range, paging, compression, and latency.

Prior equationMem_KV = B * N_layers * T * H_kv * d_head * 2 * bytes
Saved evidenceChecking saved KV history.
BoundarySmaller KV bytes do not prove reliable retrieval, cheap paging, or faster serving.
Station objectroute:ai-lab/transformers/long-context#long-context-station
Backfill the KV checkpoint

Selected object

Position-sensitive evidence inside a longer window

demo:attention-transformers/long-context#interactive-demo

A token can be inside the advertised context window and still be less usable. The local witness is a toy evaluation surface, not a benchmark claim.

Prediction checkpoint

Your KV checkpoint made stored K/V narrower. Now T grows. Which pressure must be tested first?

Manipulation instrument

Move task, position, distractors, and length without changing the question.

The score is a deterministic local witness that makes the shape of the failure inspectable. It is not a model benchmark and it should not be read as source evidence.

Nominal T
Evidence position
Task
Distractors
Choose a pressure before evidence.

Route spine

After usable context, the route becomes a workload.

The main Transformer Systems Lab path goes to serving and decoding. SSM Hybrids remains a concept branch for compact-state sequence modeling, not the primary systems route.

  1. 01KV memory

    H_kv reduced cache width at fixed T

    ready
  2. 02Long-context use

    Position, retrieval, range, paging, latency

    active
  3. 04Concept branch

    SSM Hybrids as compact-state alternative

    branch