{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfd486ba-fdc9-5a0c-bbec-5856eae3caab","task_key":"default--validation--24664","task_revision_id":"1","upstream_id":"24664","short_description":"HellaSwag validation 24664","config":"default","split":"validation","body":"{\"activity_label\":\"Croquet\",\"ctx\":\"The men then hit a ball all around the yard through stands sticking out and speak to one another while drinking water. the men\",\"endings\":\"['are in a race to score and everyone in stands runs over to beat them.', 'hit the ball a few more times and continue to speak to one another.', 'continue to speak while using all manner of beverages and drinking.', 'continue hitting the ball and are seen riding around with their dogs.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}