{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24b178ae-b286-5aba-a0df-3fac03a2170a","task_key":"default--validation--40575","task_revision_id":"1","upstream_id":"40575","short_description":"HellaSwag validation 40575","config":"default","split":"validation","body":"{\"activity_label\":\"Putting in contact lenses\",\"ctx\":\"A woman with short hair is wearing an earphone and eyeglasses is showing two clips, then she clipped both sides of her hair, showed her blue and white contact lenses containers and removed her eyeglasses. she\",\"endings\":\"['then showed the glasses with contact lenses and began to brush off the windowed lens of the lens and showed how broken nails can be.', 'started braiding her hair, she put down the contact lenses after only two tries.', 'show the screenshot and put down her donle.', 'showed her hand, then showed a sclera lenses, and put it on her right eye pupil, went closer to the camera to show her eye, then removed the contact lens from her eye by pinching it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}