{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96d2b509-d18a-5597-88e5-acbb6b0ae040","task_key":"default--validation--36741","task_revision_id":"1","upstream_id":"36741","short_description":"HellaSwag validation 36741","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make pickled chiles [title] Separate out bad chiles. [step] Because the oil from the chiles can irritate your skin and eyes, wear plastic gloves when handling your chiles and wash your hands after handling chiles barehanded. Remove limp, spotted, and soft chiles.\",\"endings\":\"['Cut them in half lengthwise using a knife, and separate the parts that are thick and that are thin. [substeps] If you prefer stronger chiles, consume the rest of the chiles.', 'Crisp, firm, fresh peppers will yield the best pickled chiles. [title] Cut the chiles and remove the seeds.', 'Wash them under cold running water if they are wild--or if you were able to remove them despite their being wild. [title] Soak the chiles in your brine.', 'Chop a small sliver and rub it on your skin. Your skin will absorb the oil much better, and you should be able to smell the pickled chiles in your mouth for as long as you touch them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}