{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"57ba1a28-c048-50fd-9d57-46292205d0be","task_key":"default--validation--18215","task_revision_id":"1","upstream_id":"18215","short_description":"HellaSwag validation 18215","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wind an automatic watch [title] Keep your arm moving. [step] The automatic watch is built with an oscillating metal weight, or rotor, that tracks movement. The oscillating rotor is attached to gears inside the watch that are in turn attached to the mainspring.\",\"endings\":\"[\\\"If your watch has an audible sound, the timing of the rotation takes the very focus away from the clock. It should not take much time for the watch to move, or unless you're using a slow moving clock on the watch.\\\", 'To wind an automatic watch, you need to keep your arm and palm facing out. This will make the watch work overtime to rotate and block the current.', 'When the rotor moves, it moves the gears which, in turn, winds the mainspring. This stores energy in the mainspring so that the watch continues ticking.', 'Monitor every single post that shoots past or crosses any of the main valves to keep track of the running time. This is important for accurate hand control and little oversight as you sleep and take taxis.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}