{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ac7b30e-ef74-5acc-a8c8-311db88c8feb","task_key":"default--validation--34834","task_revision_id":"1","upstream_id":"34834","short_description":"HellaSwag validation 34834","config":"default","split":"validation","body":"{\"activity_label\":\"Sharpening knives\",\"ctx\":\"A young boy is shown speaking to the camera and eventually shows his hands moving objects. He places the objects together and begins shaving the sides with a sharp knife. he\",\"endings\":\"['continues sharpening the object and shows the knife running across his bare skin.', 'then holds a bottle up to the camera and takes it with him so he can finish shaving.', 'then lifts up the camera and shows the bathroom and shaving cream.', 'then goes down their hands, scraping it away, and after shaving his back.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}