{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"53282c38-890c-53f9-aa8f-6372dde84872","task_key":"default--validation--12275","task_revision_id":"1","upstream_id":"12275","short_description":"HellaSwag validation 12275","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a combined engagement and wedding ring [title] Try yellow gold for sensitive skin. [step] While rose gold is renowned for its durability, some people have allergic reactions to the copper in the metal. Therefore, if the bride has sensitive skin, it's best to shy away from rose gold and go for yellow gold.\",\"endings\":\"['[title] Pick rose gold for durability. [step] Of the traditional wedding and engagement ring metals, rose gold is the most durable.', '[title] Select bright, bold colors for practical purposes. [step] While wedding rings and engagement rings can be very bold, they could come in a wide range of colors.', 'Rose gold has a much lower toasting point than red gold. [substeps] Red gold rings are bulkier and is known for its heat resistance.', \\\"[substeps] Mustard gold is great for sensitive skin. Since it contains 300 to 200% zinc (zinc nitrate), it's an ideal option for sensitive skin.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}