{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9edb841c-5a0b-532e-8428-43a246a6aa06","task_key":"default--validation--43369","task_revision_id":"1","upstream_id":"43369","short_description":"HellaSwag validation 43369","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to get wax off a tablecloth [title] Freeze the dripped wax. [step] The easiest way to do this is fold the tablecloth with the wax on the outside top layer and put the entire cloth in the freezer. Or, you can fill a bag with ice and place the ice pack on top of the wax while the cloth is still laid out on the table.\",\"endings\":\"['[title] Use a dull knife, such as a butter knife, or your fingernail to carefully scrape the frozen wax off the tablecloth. [step] If the tablecloth is made of lace use extra care not to tear the lace material while scraping the wax.', 'You can also leave the ice packs on for up to two days. [title] Heat water to 180ºf (14ºc).', \\\"[substeps] It might be a nice idea to wrap the entire tablecloth in freezer paper first. The ice will trap any dripping wax so that it doesn't accumulate much over time.\\\", '[title] Cover the bottom seam with the wax. [step] Let the wax harden a bit before you lay it out in the sunlight.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}