{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5f82062e-e073-5c4b-b0d4-91600af789e8","task_key":"default--validation--26055","task_revision_id":"1","upstream_id":"26055","short_description":"HellaSwag validation 26055","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to decorate an open plan home [title] Plan ahead for space. [step] Before you ever move your furniture in, decide what areas are most important to you. For instance, if you entertain often, you may decide a large dining area is important to you.\",\"endings\":\"['On the other hand, if your family loves to have movie nights, you might want to spend more space on the living room. [substeps] Think about the size of your furniture and how it will work in the space.', \\\"However, it's also important to plan your space well in advance. Otherwise, it's easy to overpack and move furniture overnight.\\\", 'If you want to develop a solid dining area or if you plan to build a traditional floor-to-ceiling space in your space, what areas are most important? [title] Plan ahead for heating and cooling. [step] The temperature will need to be maintained in most places in your home.', 'Also, plan for spaces that are divided enough so you can house several people comfortably. [substeps] Some large families might wish to keep private rooms for friends and family.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}