{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26346c67-ae90-50e7-b156-82ba51fbb54b","task_key":"default--validation--47807","task_revision_id":"1","upstream_id":"47807","short_description":"HellaSwag validation 47807","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to customize your cubicle at work [title] Empty your drawers and cabinets. [step] This first step is essential to \\\" de-cluttering \\\" your cubicle. You should get rid of anything that serves no functional or aesthetic purpose.\",\"endings\":\"['A little trash gets thrown out every week, broken papers pile up every few days, and items on your desk constantly disappear. Go ahead and put up your side drawers and shelves.', 'This step is especially true if you have a large project at the office that requires undivided access to your work space. [title] Have enough space for the number of folders, papers, and notebooks you will need.', \\\"Once this is done, you can reorganize the items you've chosen to keep so that they are easy to find. [substeps] You may find it helpful to assign certain drawers, shelves, or cabinets to certain categories of materials.\\\", \\\"[substeps] Don't throw old items or take them to the dumpster. This includes things like pens, erasers, and papers, saved for the changing area in your bathroom or supply closet.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}