S

SWAG

Software / AppMentioned in 1 video

Another benchmark that appears to suffer from the 'first character equals final answer' issue, similar to issues found in the MMLU.

Videos Mentioning SWAG

SmartGPT: Major Benchmark Broken - 89.0% on MMLU + Exam's Many Errors

SmartGPT: Major Benchmark Broken - 89.0% on MMLU + Exam's Many Errors

AI Explained

Another benchmark that appears to suffer from the 'first character equals final answer' issue, similar to issues found in the MMLU.