{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"966d74c8-5595-5e81-ba77-2fc3c5af0920","task_key":"default--validation--15181","task_revision_id":"1","upstream_id":"15181","short_description":"HellaSwag validation 15181","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to shake your hips [title] Stand with your feet just a bit more than a shoulder width apart, in front of a mirror. [step] This is the proper width for your feet when doing this dance move. It will help you engage your core muscles.\",\"endings\":\"['[title] Grab your feet with your left foot and lock your knees. [step] Then, bend your left knee and both your feet are bent, like a 90 degree angle.', \\\"You'll also want your knees to be slightly bent. You want to let your abs do the work when you shake your hips, not your legs or back.\\\", '[title] Gently step forward with your right leg. [step] Lean forward slightly and bringing your right foot to touch your left knee.', 'For best results, make sure your knees are bent and your big toes are firmly placed on the ground. [title] Lean forward slightly from the hips until your head is directly at and about eight inches (20.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}