{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dfdbdec5-652b-595a-baf3-93b68c72d0a5","task_key":"default--validation--8519","task_revision_id":"1","upstream_id":"8519","short_description":"HellaSwag validation 8519","config":"default","split":"validation","body":"{\"activity_label\":\"Futsal\",\"ctx\":\"A black screen is pictured and white words begin to bounce down on the screen. after\",\"endings\":\"[', the perber machine wakes up and the black is seen and white is shown across the screen.', ', a group of young men are pictured playing indoor soccer.', 'they are done bouncing, the man in blue returns and bowls it back, which continues to bounce on the table one at a time.', ', a group with short hair and an under armour appear and begin getting ready mentally for the exercise.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}