{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f39f1ddf-72c9-5f82-97ba-7c44f82ec815","task_key":"default--validation--23332--variant-f924e353bcc2","task_revision_id":"1","upstream_id":"23332--variant-f924e353bcc2","short_description":"HellaSwag validation 23332","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to remove wrinkles from a tie [title] Undo the knot. [step] It is important to untie the knot after you are finished wearing it. Undoing the knot prevents permanent creases, and it is the first step in removing existing wrinkles.\",\"endings\":\"['First, loosen the knot by gently pulling it from side to side. Then, pull out the short and long end of the tie with your thumb.', '[substeps] The knot may come loose as you pull the tie over your head. It might also drop to the floor while you are looking in the mirror, so to avoid this risk, try to undo the knot more effectively.', '[substeps] Untie the knot just below the knot. This will open up the knot but does not remove the wrinkle.', 'When the knot was undone, it will be evident that you have undone the tie. Gently remove the knot while loosening the shirt to reveal the intricate knot.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}