{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fec24a5b-f674-5b23-8c3e-52f998d956b9","task_key":"default--validation--11348","task_revision_id":"1","upstream_id":"11348","short_description":"HellaSwag validation 11348","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A man is outside in a field of grass pushing a lawn mower back and forth. once he\",\"endings\":\"['is done, the man takes off so fast the lawn mower resumes mowing again before stopping and holding out his hand and ending the video.', 'is done he turns around and starts in on another bull.', 'makes it to the other side of the strip, the person with the camera walks closer to him and starts to talk to them.', 'is finished, a smaller blue mower comes out and it is pushed away from the lawn.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}