{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac8dc919-a153-5028-a284-6293d56713b7","task_key":"default--validation--39394","task_revision_id":"1","upstream_id":"39394","short_description":"HellaSwag validation 39394","config":"default","split":"validation","body":"{\"activity_label\":\"Clipping cat claws\",\"ctx\":\"A person is seen close up cutting the hooves of a pig and moving his hands around to see. the person\",\"endings\":\"['continues cutting the hooves as well as checking the teeth.', 'continues cutting through the cuts on the horse and speaking to the camera.', 'peddles the pigs and points to the hooves and continues cutting them over and over.', 'is then seen again inside a building brushing the front of a truck with the electric shaver.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}