{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02a0fbf3-b2bd-524e-afa5-5cc30337c45d","task_key":"default--validation--22223","task_revision_id":"1","upstream_id":"22223","short_description":"HellaSwag validation 22223","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to organize your office [title] Sort your papers. [step] It's easy to let paperwork pile up on your desk, but this ushers in a sense of clutter, which bogs you down. Get a filing cabinet, small paper organizer with shelves for your desk or some other type of file holder to sort your papers into.\",\"endings\":\"[\\\"Once you're done with something, file it, shred it or throw it away. If you have to keep a document but have no use for it at the present time, put it in storage.\\\", \\\"[substeps] Sorting papers not only takes up space but also forces you to prioritize your papers so you start the process as soon as they're sorted. Anything that takes up space will end up failing to be accessible in your filing cabinet.\\\", \\\"If you're looking for specific papers, select one of those for your pile. [substeps] Sorting papers by type will make organization quicker and easier, as you will get to have all of the papers in one place.\\\", '[substeps] Group your pens, pencils, sharpeners, erasers, and other office supplies. One stack will probably be enough for a big pile of paperwork.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}