{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f4cfefd-b20c-5912-b20b-68b54b3be5b0","task_key":"default--validation--32673","task_revision_id":"1","upstream_id":"32673","short_description":"HellaSwag validation 32673","config":"default","split":"validation","body":"{\"activity_label\":\"Ping-pong\",\"ctx\":\"As they both rise we see that they are actually heading to a table tennis game, the table on an actual tennis court. they\",\"endings\":\"['being their match, one in a white tank top and green shorts, the other in an orange shirt and white shorts.', 'take their key cards and go over to join other players.', 'begin to go through the game and a white ball lands in a net while the single tennis player in pinata style stands over it with his hands up.', 'are playing doubles doubles, and using two sticks to hit balls.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}