Multi-Token Prediction
Concept
An approach where the main LLM itself is used as both the draft and target model, training K heads to predict future tokens simultaneously, leveraging the first head as the verifier.
Mentioned in 1 video
An approach where the main LLM itself is used as both the draft and target model, training K heads to predict future tokens simultaneously, leveraging the first head as the verifier.
