{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e66370d-8d26-50d4-b269-d3f61bd69521","task_key":"default--validation--20987","task_revision_id":"1","upstream_id":"20987","short_description":"HellaSwag validation 20987","config":"default","split":"validation","body":"{\"activity_label\":\"Ballet\",\"ctx\":\"The teacher instructs the girls how to perform proper ballet moves, as the students follow. The girls move their arms in the correct position and eventually turn to one side. finally the instructor\",\"endings\":\"['instructs their steps and moves as their lesson begins.', 'stops the dance and the girls turn so side to side.', 'teaches the girls to extend their legs and perform a proper move.', 'explains how to properly perform solo moves.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}