{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3547b154-2cb7-5ac1-bb75-0e765974a1d8","task_key":"default--validation--29786","task_revision_id":"1","upstream_id":"29786","short_description":"HellaSwag validation 29786","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to spot fake nmd shoes [title] Look for deep blue laminated letters that spell boost on the box. [step] Turn the closed box so the white label on one end is facing you. On real nmd shoes, the oversized word \\\" boost \\\" appears, starting with the b and o on this panel of the box and wrapping around.\",\"endings\":\"['[title] Look for white stamped shoes with other labeling patterns. [step] These patterns might be labeled \\\" special brand \\\" or \\\" medical brand \\\" or simply simply stenciled on the box.', '[title] Examine the lock number on the other side of the box. [step] All letters currently spelled boost should have a letter written to the left of the lock number as its letter.', 'You may also see extra underlined writing beneath the letters. Look for luster or gloss on the back of the shoe.', 'The word appears in a shiny, slightly darker shade of dusty blue than the surrounding background color. [substeps] Fake nmd shoes often start the word boost on another panel or in the wrong color.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}