{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cad4eb76-6399-5ecf-806b-88a49c6aedaa","task_key":"default--validation--35602","task_revision_id":"1","upstream_id":"35602","short_description":"HellaSwag validation 35602","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to stretch your biceps [title] Stand up straight with your feet positioned hip-width apart. [step] If preferred, you can stand with your back against a wall or pole to deepen the stretch. [title] Bend your knees slightly and point your toes forward.\",\"endings\":\"['[title] Lift both arms straight out to either side at shoulder height. [step] Your shoulders should remain even with one another.', \\\"[step] Keep your toes pointed until you feel a stretch in your biceps. [substeps] Don't point your toes as you shift your weight to the balls of your feet.\\\", '[step] If you prefer to lie down on your back, extend your elbows to about eye level. The backs of your knees should be very close to one another and the tops of your feet should be slightly inches from one another.', \\\"[step] You may want to vary the positioning of your feet if you're sitting so you can stretch your biceps more deeply. [title] Bring your toes in towards your buttocks.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}