{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7f7dea2-5a5b-5d57-ac3b-8080e69dce8d","task_key":"default--validation--5072","task_revision_id":"1","upstream_id":"5072","short_description":"HellaSwag validation 5072","config":"default","split":"validation","body":"{\"activity_label\":\"Beer pong\",\"ctx\":\"A person moves the cups alongside the table while the person continues throwing and the men switch place. the two\",\"endings\":\"['men talk, then the man puts a fountain pen on the cup and begins talking.', 'begin walking on the bridge again and the sexy figure moves the cups on table, while more persons switch places, and they all move their cups around, intermittently, with the woman in the middle.', 'starts playing alongside each other with a plate lined up.', 'continue taking turns while the camera captures their movements.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}