Logits
Concept
The raw output scores from a model before the final activation function, used in distillation to mimic the teacher model's decision-making process.
Mentioned in 2 videos
Videos Mentioning Logits
The Unreasonable Effectiveness of Reasoning Distillation: using DeepSeek R1 to beat OpenAI o1
Latent Space
The raw output scores from a model before the final activation function, used in distillation to mimic the teacher model's decision-making process.
Personal benchmarks vs HumanEval - with Nicholas Carlini of DeepMind
Latent Space
The raw, unnormalized output of a language model before the softmax function; exposing logits can enable model stealing attacks.