{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8523355b-6c8e-535f-b39d-bf4a5fc8c058","task_key":"default--validation--9997","task_revision_id":"1","upstream_id":"9997","short_description":"HellaSwag validation 9997","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a creme blush [title] Take your usual powder blush or eyeshadow compact and have the brush in your hand. [title] Tap the blush with the end of the brush, using the bristles end. [step] You should end up freeing a layer of \\\" loose \\\" powder makeup.\",\"endings\":\"['Use your fingers to separate the powder colors (gray, black, blue, yellow) from the blush. If you plan on using another color, hold the translucent powder (in a separate plastic bowl) into the palm of your hand, with the edge facing upwards.', '[title] Dip the brush into the blush, using the center for the shade you want your blush to be. [step] Hold the brush at the corner of the brush, to the shade.', 'You will see much of that powder on the brush and on the pads of your fingers. Get creative and fill in your cheeks with the powder makeup, how you wish.', '[title] Flip the compact upside down over a dish. [step] Makeup granules should fall onto the dish.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}