{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8fffa98-2011-5700-9832-a36b374f1c72","task_key":"default--validation--49643","task_revision_id":"1","upstream_id":"49643","short_description":"HellaSwag validation 49643","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to create a fairy garden [title] Select a container. [step] Fairy gardens are usually housed in small containers. Think about how much space you have.\",\"endings\":\"['[substeps] You can place each fairy garden in a bowl with water at one end or in a container full of larger objects on either side. Fairy garden containers come with multiple sides to hold various fruits, vegetables, herbs, spices, trees, and flowers.', 'The size of a three-gallon bucket will be best for the fairy garden. The container you choose should also accommodate a few larger fruits and vegetables.', 'You may want to place one container in each room and the other in the room itself, so that the fairies can see each other. Think about how much room you need to accommodate the other fairies.', \\\"If you're keeping your garden indoors, you'll need a smaller container. For an outdoor garden, you can use something larger.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}