Decoding Methods Comparison

Each decoding method's final answer is printed as soon as it is done. Model used: HuggingFaceTB/SmolLM2-360M-Instruct.

Prompt

Greedy

Top-k Sampling

Top-p Sampling

Beam Search

Min-p Sampling (as in https://arxiv.org/abs/2407.01082)

Eta Sampling

Epsilon Sampling

laconic decoding (Alex Dimakis: https://x.com/AlexGDimakis/status/1885447830120362099)

Power Sampling (Karan & Du, 2025; beta=2, 4 MCMC steps)

COT Decoding (Chain-of-Thought Reasoning without Prompting, Wang, Zhou, 2024)