{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72778515-3a88-51b9-811b-4ac21f03f17a","task_key":"default--validation--25037","task_revision_id":"1","upstream_id":"25037","short_description":"HellaSwag validation 25037","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make an elastic band \\\" braid \\\" [title] Prepare your hair. [step] Brush your hair and spray a little dry shampoo at the roots. [title] Pull your hair into a ponytail.\",\"endings\":\"['[step] The ponytail should be about 1 inch (2.5 cm) high so you will have enough hair to wrap around your wrist at this point. You can also skip braiding and just tie it around your scalp instead.', \\\"[step] It is easier to do this to the side if you are doing your own hair. If you are doing someone else's hair, consider doing it at the back.\\\", '[step] Consider braiding in three sections into a neat ponytail. [substeps] If your hair is very coarse use all three sections.', '[step] Alternatively, if you have naturally straight hair, use your bangs as an alternative. You can use either a neck scarf or a bandana.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}