{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28f7858f-fb4b-5eb0-8714-1864df15f38d","task_key":"default--validation--41168","task_revision_id":"1","upstream_id":"41168","short_description":"HellaSwag validation 41168","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to teach a child to use scissors [title] Take a pair of scissors and hand the child the handle side first. [step] Naturally the child will pick the pair of scissors with their dominate hand. [title] Start with the grip.\",\"endings\":\"[\\\"[step] Show the child in your hand how to hold the scissors, and then have them try to copy it. If they aren't able to alone, help move their fingers and thumb into the right position.\\\", '[step] Hold the handle side in one of your dominant hand, use thumb and pointer fingers to grip the grip side of the scissors, then use the second pair of scissors to cut away any excess skin. If you want to do this instep the toe with your palm.', \\\"[step] Holding the handle side of the scissors firmly between your thumb and forefinger, swiftly bend the handle and snip off a small piece off the short end. [title] Grip the blade using your dominant hand and hold it at a 45-degree angle from the child's fingers.\\\", \\\"[step] This is the grip you can get to hold the child's scissors so as to position them correctly. [substeps] Hold the teacher's scissors straight up.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}