{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0ac60f86-9d07-5a28-b4a2-46e84aadab86","task_key":"default--validation--35915","task_revision_id":"1","upstream_id":"35915","short_description":"HellaSwag validation 35915","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be the girl everyone wants to be [title] Get organized and stay that way. [step] Being perfectly put together is a quality many people will envy. Start by cleaning your room to perfection, and making sure there is a place for everything and everything has its place.\",\"endings\":\"['Also be sure to get rid of things you know you don\\\\'t need. Minimizing your \\\" stuff \\\" will help you find things quicker and easier.', \\\"You will be assured that you haven't forgotten anything in your room. Be careful not to take things haphazardly.\\\", \\\"Organized girls would have laptops, notebooks, liquids, and sharpies. You might find that all you have to work with is an empty desk or a messy binder, just keep it organized and don't have yourself responsible for all the work right now.\\\", \\\"If you feel your room is cluttered or disorganized, don't attempt to change it. Cleaning is an impossible process, so you want to avoid this, and to prepare yourself for it physically.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}