{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5f2cf169-deac-5b32-b3a2-01f9555814dd","task_key":"default--validation--14589","task_revision_id":"1","upstream_id":"14589","short_description":"HellaSwag validation 14589","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to cut curly hair [title] Get your supplies. [step] You'll need a good pair of cutting scissors. They should be very sharp, as dull scissors won't give you a good cut.\",\"endings\":\"['A scissors grinder or a sewing machine will be fine, just the blade and paper towel. [title] Get a section of hair that you want to cut.', \\\"You'll need several sheets of paper that each measure about an inch and a half in length. These are the good pieces to start with so that your cut is easy to figure out.\\\", \\\"You'll also need hairpins, clips, or hair ties if you want to section off part of your hair, as well as a wide-toothed comb. [substeps] Only use your hair-cutting shears for cutting hair.\\\", 'Look for scissors that have rounded sides with a pointed tip. They can also get dull once they come to a stop.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}