-
ASD: Trading a Bounded Regret Budget for Faster Speculative Decoding
Design notes on a bounded-regret acceptance policy for speculative decoding — +13.3% tokens/s on Qwen3-14B at a cost of 0.23 pp accuracy on GSM8K.
Design notes on a bounded-regret acceptance policy for speculative decoding — +13.3% tokens/s on Qwen3-14B at a cost of 0.23 pp accuracy on GSM8K.