{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a06cc9f0-55cb-5255-8401-4ffa7f9ac861","task_key":"default--validation--15144","task_revision_id":"1","upstream_id":"15144","short_description":"HellaSwag validation 15144","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to draw yoshi from mario [title] Draw a couple of circles, one 80% smaller than the other and overlap those. [title] Add an angular oval on top of the bigger circle and add an overlapping circle at the bottom of the oval and another horizontal oval touching the base of the circle. [title] Add details to the head by drawing the eye, eyelid and the nostril.\",\"endings\":\"['[title] Add details like the golden boots and the nose along with a hat. [title] Do a craft and then outline the head and add a detailing to the hat.', '[title] Add two small circles at the mouth and overlap them a bit but keep the one at the base going into the throat. [title] Draw the shoulders and red cheeks and/or a couple of spikes from the cheeks.', '[title] Draw a crescent moon-shaped sketch for the body of the yoshi. [title] Draw a circle on the concave centre and another smaller one inside it.', \\\"[title] Add facial features by drawing the billowy hair on yoshi's head. [title] Erase some but don't erase too many details.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}