{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f16e3e2-fd6c-5f3c-b3d2-0f9319c5256d","task_key":"default--validation--9957","task_revision_id":"1","upstream_id":"9957","short_description":"HellaSwag validation 9957","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to learn cad design [title] Download a free software. [step] Find a software that is free to start learning there are plenty of them out there. Such like blender or autocad [substeps] If you are not sure what software is suitable for you, can try the trial version.\",\"endings\":\"['The us has integrated cad visual fabrication, such as cbp on all computer computers. Pay more attention to software like wordpress, audacity, metro and google docs.', 'If you are new to this, try a simple tutorial. Otherwise, try searching internet website, and then see if a user you know has a free program waiting for them.', '[title] Once we have the software in hand, give it a try. [step] Try all the features, try the primitive shapes (cubes, spheres, etc), the tools and the rendering.', 'Versions like adobe, microsoft word, and microsoft excel will help you understand your work and your web interface for several weeks. [title] Acquaint yourself with your operating system.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}