{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25628f4a-73ad-59c0-aac2-9b21b4c85a6b","task_key":"default--validation--44565","task_revision_id":"1","upstream_id":"44565","short_description":"HellaSwag validation 44565","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to buy an urn [title] Choose the material for your urn. [step] If you plan to keep the urn of ashes for display, then it would be best to purchase an attractive, yet durable vase. Consider the area and the room in which you plan to display the piece, as this may influence the material you choose.\",\"endings\":\"['Think about how you will be placing the urn, and how much storage space will you need. [substeps] Considered assessing the noise level and the décor in the room where you plan to display your urn.', 'For example, urns can be made of many different substances like wood, stone, ceramic, metal, or glass. [substeps] If you are burying the urn or placing it in a mausoleum, keep in mind that you may be limited in the material you choose.', '[substeps] For example, you may choose a urn made by an artist or another representative of your community. Another example is if you plan to display an urn of glow sticks (from a trusted source), you would definitely want this vase.', '[substeps] If the urn is dry, you may choose to use a gentle and breathable material, such as silk, made from natural fibers. If it is hard or prickly, consider choosing a very moisture-wicking material, such as velvet or plaster.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}