{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e606a5e5-0d9e-5506-9f93-68e154e08726","task_key":"default--validation--18519","task_revision_id":"1","upstream_id":"18519","short_description":"HellaSwag validation 18519","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become a cartographer [title] Understand the job description. [step] All cartographers must be able to visualize data using both analysis and artistic imagination. They use a close attention to detail and accuracy in order to translate the data into a map, and have a reputation for perfectionism.\",\"endings\":\"['In general, cartographers must understand how to solve problems properly. [substeps] You want to come up with a specific category for your current cartography talents, then put these skills into action.', \\\"Sketchacosee is a master cartographer, who uses these qualities to develop and arrange detail, and has a creative mind that critiques images, scenes, and ideas. Whenever you're researching how to become a cartographer you'll have to pay attention to how people would view your writings.\\\", 'Computer programming skills and mathematical ability are increasingly valued in the cartography job market, but you do not necessarily need advanced experience in these subjects. [substeps] Most cartographers have office jobs, and spend 35-40 hours a week working mostly on the computer and drafting table.', 'For cartographers of all ages, the career paths, licensing, and skills are often discussed in private schools or university campuses. [title] Decide if the career path is appropriate for you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}