{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d278fad0-e6b4-51f5-8713-8504442ca19f","task_key":"default--validation--33939","task_revision_id":"1","upstream_id":"33939","short_description":"HellaSwag validation 33939","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to shape eyebrows [title] Determine where your inner brow should end. [step] Hold a straight edge, such as an eyebrow pencil or a ruler, vertically in front of your face. [substeps] If you want your eyebrows farther apart, line it up so that it touches the outermost of edge your nose and the inside of your eye.\",\"endings\":\"['For more precise shading, inline your brow with the fine tipped tip tip of a pencil. [title] Sketch two lines horizontally and vertically, with the brow almost touching them.', 'If you want your eyebrows higher and thinner, the lines at the outer edge should stay in place at the outer corners of your eyes, creating another arch. Another method, if applicable is just to close one eye over the corners on either side of your nose.', 'If you want a straighter looking eyebrow, better pencils should line up on the inner edges of inner brows as well. They should also cross at the corners so that the upper is longer than lower to create a sharp angle.', 'If you want your eyebrows closer together, line up the straight edge with one side of the septum. These lines will determine where your eyebrow should start.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}