FastPathLabs Research in LLM inference

Experiments

Each experiment is written up as a research note: the question, the method, the result, its limits, and the references it rests on. Notes are published whether or not the result cleared its bar.

  1. Research note 1

    10 November 2026

    Engine settings on a competent baseline: a pre-registered search

    Thirty combinations of four vLLM settings, measured on one H100. The best raised goodput by a factor of 1.14 against a bar of 1.5, so the experiment stopped. Two settings produced the whole gain.

    Status
    Completed
    Hardware
    1 × H100
    Model
    Qwen3-8B, FP8

  2. Research note 2

    10 November 2026

    Cache-aware routing for multi-turn chat

    A router that returns each turn to the worker holding the conversation, compared with round-robin and least-loaded routing on four GPUs. The gain is real, and bounded by the prefill that would otherwise be repeated.

    Status
    Preliminary
    Hardware
    4 × RTX A6000
    Model
    Qwen2.5-7B

  3. Research note 3

    11 November 2026

    Speculative decoding: tokens per verification and the cost of a round

    On a small laboratory model, fewer forward passes per token did not become more tokens per second until the whole round was compiled into one graph. The note separates the small-model and large-model regimes and gives the design of the run on an H100.

    Status
    Laboratory result; GPU run specified
    Model
    16.7M-parameter laboratory model

  4. Research note 4

    13 November 2026

    Recurrent-state quantization for linear attention: a first GPU kernel

    An implementation of LeapQuant, which stores the recurrent state of linear-attention layers in 8 bits. The reference reproduces the arithmetic. The first kernel is 7.2 times slower than SGLang’s window kernel, and the note shows where the time goes.

    Status
    In progress
    Hardware
    1 × Hopper-class GPU
    Model
    Qwen3.5-9B

  5. Research note 5

    10 November 2026

    Measuring goodput: calibration and four guards

    How capacity is defined, what calibration showed about throughput and attainment, and four faults in our own measurement that were each larger than the effects under study.

    Status
    In use
    Hardware
    1 × H100
    Model
    Qwen3-8B, FP8