{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2caf92d7-c6a6-5569-80bd-90197cbba610","task_key":"default--validation--8879","task_revision_id":"1","upstream_id":"8879","short_description":"HellaSwag validation 8879","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use hand sanitizer [title] Clear your hands of all debris and jewelry. [step] Take off all rings and other jewelry that may be covering the surfaces of your hands. If possible, rinse and remove all traces of visible organic matter such as dirt, grease, and food in order for the hand sanitizer to be most effective.\",\"endings\":\"['[title] Squirt the hand sanitizer into the palm of one hand. [step] Be generous with the amount of sanitizer applied.', 'You may also consider dry cleaning items such as oven mitts and a dust mop. [substeps] Wash your hands before and after handling or using the hand sanitizer.', '[substeps] This is especially important if you have blackheads or other serious issues. Normal hand sanitizer will not provide an effective solution.', '[substeps] If you want to avoid contaminating your hands with harmful germs, turn your hand sanitizer gel into a face wash first, and dry your hands with a soft towel. Always shake the cap off before rinsing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}