{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24e680a2-c7b9-5023-969d-43c54fcf6bcf","task_key":"default--validation--10820","task_revision_id":"1","upstream_id":"10820","short_description":"HellaSwag validation 10820","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a parka [title] Go with a lightweight parka on chilly days. [step] If you want to stay warm on slightly cold days, go for a parka that isn't super puffy and insulated. A thin, lightweight parka will keep you warm on cold days that don't go below freezing.\",\"endings\":\"['[substeps] For instance, wear a parka with a thick sweatshirt or jacket over it. [title] Select a thick parka if you want an elaborate fit for a party.', 'Select a parka with 50-100 g of insulation. [substeps] Try a lightweight parka if you are running errands or taking a jog, for example.', 'Winter is one of the more temperate seasons in the u.s., so you can still go for a parka no matter what temperature it is. [title] Wear your parka on very hot days.', '[substeps] Temperatures that fall below freezing are often determined by how dark the weather outside is in the middle of the year. For example, if the temperature is warm in the winter and lows remain only slightly below freezing in autumn, bring your parka over.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}