{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97d2f066-2365-5565-a2bf-3476f3ac6d54","task_key":"default--validation--35878","task_revision_id":"1","upstream_id":"35878","short_description":"HellaSwag validation 35878","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to start learning japanese [title] Learn hiragana. [step] Hiragana is the japanese alphabet. It is comprised of 51 phonetic characters, and each character stands for exactly one sound.\",\"endings\":\"['Every japanese letter has their own phonetic sound. Hiragana is one of the angriest sounding parts of the japanese language.', '( this is different from english in which one letter may sound different in different contexts). Once you know hiragana, you will know how to pronounce any word in japanese.', 'Learning hiragana will enable you to learn more every day. To do this you should first try learning it on your own.', 'Hiragana is often used to make handitions, as well as to make performances. The pronunciation of the letters vary, and it is important that you learn the correct pronunciation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}