{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f50bcc45-bcd1-57f0-904e-49645da785e6","task_key":"default--validation--361","task_revision_id":"1","upstream_id":"361","short_description":"HellaSwag validation 361","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to pluck eyebrows without pain [title] Heat up some water. [step] The easiest way to heat up water is to fill a mug halfway up with water. Put it in the microwave for about 30 seconds.\",\"endings\":\"[\\\"You don't want to get the water too hot, as that could burn your hands or face. You'll also need a washcloth to apply it to your face.\\\", \\\"The hot water will make your eyebrows soft, meaning they'll become easier to pluck. Cover the mug and put on some cotton or plastic wrap to protect it from the water.\\\", 'Then, make sure you quickly place it under the spigot. You can also try soaking your eyebrows overnight in the water.', \\\"The water should be hot and not make you too hot. [substeps] Don't microwave water for too long.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}