{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9da6b0ad-2624-5fe7-81a9-f0f1776ab48f","task_key":"default--validation--42149","task_revision_id":"1","upstream_id":"42149","short_description":"HellaSwag validation 42149","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to do a fast, deep clean for your bedroom [title] Divide your room into zones. [step] There should be a sleep zone (a bed), a clothes zone (a closet or dresser), a get ready zone (a mirror and makeup, jewelry, etc. ), a relaxation zone (music player, games, magazines), and a work zone (a desk or work table).\",\"endings\":\"['These rooms should be about equal distances, though you should also separate of them a bed zone and a work zone. [substeps] Set the entire room to clean if you want to.', 'Arrange these zones within your living area and work or schoolwork zone so that they will seem more accessible to you. You may need to allot a bit of time in each zone to relax and rejuvenate.', '[title] Work on one zone at a time, deep cleaning each one. [step] This zone is easy to clean and organize.', '[substeps] Not everyone sleeps or spends the night in a zone. [title] Place a couple belongings strategically around your bed.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}