{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ae7058b0-7939-5555-b6b1-0268418191f2","task_key":"default--validation--13435--variant-a02cca8b876d","task_revision_id":"1","upstream_id":"13435--variant-a02cca8b876d","short_description":"HellaSwag validation 13435","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to sharpen an axe [title] Protect your hands and face. [step] Take these steps to avoid injury while sharpening an axe : [substeps] Wear thick leather gloves. If your file does not have a finger guard, cut a slit in a piece of leather and fit it onto the file.\",\"endings\":\"[\\\"The slit should be close to your body's surface. Insert your nail into the slit, then stop as you slowly edge your nail away from the sharpened edge.\\\", 'Secure the eye guard in place with butterfly clip. Hold the axe with one hand with the upper half of the axe alongside the lower half.', 'Wear safety goggles to protect against metal dust. A dust mask is optional for hand sharpening.', 'Wear a wide-brimmed hat or a thick halloween hat to protect your face and just cover your hands. Sweat away from using large dark colors, including black, gray, and dark orange.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}