{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df928b05-fe61-5df8-9425-7041c8918644","task_key":"default--validation--2351","task_revision_id":"1","upstream_id":"2351","short_description":"HellaSwag validation 2351","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to program on the go [title] Determine the locations at which you would ever want to program. [step] On a train would be a good idea, but not necessarily a chip stand. This is an important step to acknowledge because being able to keep location in mind will help you to envision your complete setup.\",\"endings\":\"['If you do decide to use a chip stand, make sure it is well-built and relatively small. A chip stand is easy to install on your train, create a link with your contact information, and access your system.', \\\"[title] What are you going to be programming? [step] If your developing apps for an ios or android device, what gear you have and software you'll be using will be affected by such a choice. If you're simply programming small business applications you may be able to get away with a smaller setup.\\\", 'Decide whether you want a chip stand or an outside, suitable location. Consider whether you wish to have your chip stand visible at all times, and if so, where.', 'Additionally, gravitate towards the places you want to train, and make plans accordingly. [substeps] Find one that suits your needs and budget.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}