{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"10b8ac53-4091-51c5-841c-417eac8e399d","task_key":"default--validation--41355","task_revision_id":"1","upstream_id":"41355","short_description":"HellaSwag validation 41355","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to set an armitron watch [title] Set the time on a standard or calendar watch. [step] The standard armitron model only tells the time. To change the time, pull out the crown (side dial) until the hands stop moving.\",\"endings\":\"['Tap the hour/minute combination to set the value. Then, go back to the crown (side dial) until the hands stop moving.', 'Keep your watch in the same position, but slow down and punch a hole where the crown mixes in with the dial if you feel that the dial will be thrown off when you hold it to avoid bumping it. [substeps] Leave the crown clock at the setting you set earlier in the year.', 'Press ctrl + r to change the hour. You must set the time to 18:00.', 'Rotate the crown clockwise or counterclockwise until the hands reach the desired time. Return the crown to its original position by pushing it back in towards the watch face.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}