{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7746c92-a5c7-5549-8279-1d1660046b8d","task_key":"default--validation--27291","task_revision_id":"1","upstream_id":"27291","short_description":"HellaSwag validation 27291","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress for fall [title] Plan on layering your clothes. [step] Temperatures fluctuate greatly during the fall. Mornings will be chilly, afternoons will be hot, and evenings will be cool again.\",\"endings\":\"['Make sure you wear your layering clothing with all of the front layers on and all of the back layers on. A neutral top sheet, wrap, or sweater is ideal, for both \\\" feel-good \\\" and \\\" cold weather.', 'If you are at work or at school all day long, you may not have the time to change your clothes. The best way to manage this is by wearing layers, which you can take off as the day grows warmer.', 'Consider wearing layers as necessary, especially for cold or wet weather. [substeps] A sweater, wrap, or jacket will keep you warm during the colder days.', 'A long-sleeved crop top with a tight, thermal underlayer is a great idea. Whenever possible, opt for one with a handful of loose mesh leggings and a cardigan.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}