{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c048a76a-6ead-5028-a94e-afca7bb04ad1","task_key":"default--validation--17197","task_revision_id":"1","upstream_id":"17197","short_description":"HellaSwag validation 17197","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do harajuku makeup [title] Change your eye color, if you wish. [step] If you have colored contact lenses to change your eye color you can put them in before or after you do your makeup. This step is optional.\",\"endings\":\"[\\\"[title] Look for any redness or swelling in your nose/brow area. [step] If you have done this before, you'll be aware that there will be a noticeable pop when you look at your lenses.\\\", \\\"If you don't want to change your eye color, you can wear something in the shade that is called your coloured contact lens. [title] Sit quietly throughout the session, without interacting with other people.\\\", \\\"[substeps] It is recommended to put them in before so that you don't smudge your makeup. [title] Start off with a base foundation at least one shade lighter than your natural skin tone.\\\", 'If your eye color is yellow or hazel, you may wish to put them in with your eyeshadow. [title] Put the eyeshadow on your eyelids (eye side, the lid side).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}