{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"943a238c-0a3d-5213-9217-d3667dabb9a9","task_key":"default--validation--32674","task_revision_id":"1","upstream_id":"32674","short_description":"HellaSwag validation 32674","config":"default","split":"validation","body":"{\"activity_label\":\"Ping-pong\",\"ctx\":\"They being their match, one in a white tank top and green shorts, the other in an orange shirt and white shorts. they\",\"endings\":\"['are both throwing the bats in the air.', 'are still caught at the marker holding onto the bumper cars.', 'hit back and forth for some time, the enjoyment of the game seeming more important the winning, and they slap hands at the conclusion.', 'both throw a shot put at each other.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}