{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb27c51d-ed33-5a38-a044-9d795a358da4","task_key":"default--validation--19379","task_revision_id":"1","upstream_id":"19379","short_description":"HellaSwag validation 19379","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do 3d nail art [title] Create two oval shapes. [step] Dip your brush into the acrylic liquid first and then into the acrylic powder to form an acrylic ball. Place the ball on the nail or on a nonstick sheet of plastic or on a piece of aluminum foil.\",\"endings\":\"['Use the brush to shape the ball into an oval. Then, use the brush to form the rounded edges into two pointed ends.', \\\"[substeps] You can use large flat cardboard for the oval shapes. Wrap your fingers around the oval shape so it's the same size as the ball.\\\", 'Make sure each of the tiny shapes is lined up and held in place by about two inches of glue or tape. [substeps] You may add more glue or tape if you need to.', 'Dip the ball one half of the way to the glittered top of the cylinder by wetting the tip and then carefully brushing off the clump. [title] Coat the glitter all around the cylinder.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}