{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a052ced1-ad15-59e1-8c83-fc159889c73e","task_key":"default--validation--27209","task_revision_id":"1","upstream_id":"27209","short_description":"HellaSwag validation 27209","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to get rid of tobacco smell fast [title] Throw away cigarette butts and ash. [step] Cigarette butts, half-smoked cigars, and ash can leave a strong lingering smell of tobacco indoors. Throwing the butts and ashes into indoor trash bins will only transfer the smell into another room.\",\"endings\":\"['If you must throw away these items, make sure they go in a trash bag safely. [substeps] If you are using a dumpster for human waste, make sure you throw out all tobacco-related trash you find in the dumpster.', \\\"[substeps] As smoking stops, the smells will significantly evaporate from the room. You'll have to account for the temporary smell and let the smoke immediately return.\\\", 'If the smell still persists, throw away the ones, and stash the equivalent of a lung. [substeps] There are multiple options for accidental combustion when you smoke a cigarette.', 'Dispose of the butts and the ashes in a bag and move the bag outside. This will eliminate some of the initial smell.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}