{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"545c52eb-4cc8-5cb7-91b0-637ef27117ce","task_key":"default--validation--18757","task_revision_id":"1","upstream_id":"18757","short_description":"HellaSwag validation 18757","config":"default","split":"validation","body":"{\"activity_label\":\"Doing a powerbomb\",\"ctx\":\"Two of the boys pick the little girl up and prop her on the shoulders of the boy wearing a white t-shirt. the boy wearing the white t - shirt\",\"endings\":\"['cuts the hair of the little girl showing his phone number and talking to the camera.', 'picks the girl up and throws her to the floor.', 'then lets her fall backwards onto the bed softly.', 'collides; the boy wearing the white t-shirt falls to the ground and people jump over them to get to the boy wearing the yellow t-shirt.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}