{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6e749a71-c73b-592f-82c9-a2cc0c05d222","task_key":"default--validation--7432","task_revision_id":"1","upstream_id":"7432","short_description":"HellaSwag validation 7432","config":"default","split":"validation","body":"{\"activity_label\":\"Making a cake\",\"ctx\":\"When the cake is all done the camera pans the whole cake so the details can be seen and once again we see the smaller scale cake and the toy. a white screen\",\"endings\":\"['appears with black letters below and we see a white screen with white words on it at the bottom of it and it says how to get it from the register with an online purchase screen.', 'appears with his logo on the left, and all of his detailed information for his shop and contact information on the right.', 'appears and words appear on the screen that say \\\" jackpot eels \\\".', 'appears and then starts moving, the drawings again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}