Logits
Concept
The raw output scores from a model before the final activation function, used in distillation to mimic the teacher model's decision-making process.
Mentioned in 2 videos
Save the 2 videos on Logits to your own pod.
Sign up free to keep building your knowledge base on Logits as more episodes are added.
Videos Mentioning Logits
The Unreasonable Effectiveness of Reasoning Distillation: using DeepSeek R1 to beat OpenAI o1
Latent Space
The raw output scores from a model before the final activation function, used in distillation to mimic the teacher model's decision-making process.
Personal benchmarks vs HumanEval - with Nicholas Carlini of DeepMind
Latent Space
The raw, unnormalized output of a language model before the softmax function; exposing logits can enable model stealing attacks.