{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c6c6ba3-5fbe-5527-b38b-836281387895","task_key":"default--validation--18534","task_revision_id":"1","upstream_id":"18534","short_description":"HellaSwag validation 18534","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to design a handbag [title] Read the latest magazines. [step] Fashion magazines will obviously have lots of full-page advertisements featuring handbags. But don't forget to look at other kinds of magazines.\",\"endings\":\"[\\\"Lifestyle, celebrity, and even home magazines may show handbags that catch your eye. [substeps] You don't have to limit yourself to the most recent magazines.\\\", 'Narrow your focus to design cohesive documents and stuff your handbag with them. For example, if you know you plan to go out once a week, expand on how you plan to spend the rest of your week and then specify which ones you will be organizing.', \\\"Strong cataloguing means you won't waste time and money just looking at magazines that don't show well. Look for sporting events, which you may wish to attend, and consider organizing the event for them.\\\", \\\"Aim for glossy glossy cover ads-because they'll tell you what it does, and how people will use the product! Modern magazines also contain searchable search engines for your favorite designers and the pictures on those pages. [title] Look for scientific papers written by experts.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}