{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1b7f10a-b75c-555d-8424-0ca6823baaf1","task_key":"default--validation--13657","task_revision_id":"1","upstream_id":"13657","short_description":"HellaSwag validation 13657","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy a badge making machine [title] Define your purpose for making badges. [step] If you want to produce badges to identify the wearers, such as employees of a small company or attendees at an event, you probably want a badge making machine that produces rectangular badges. If you are making badges that will be worn day in and day out, you'll want to be able to laminate them.\",\"endings\":\"['There are specific regulations and procedures regarding badge making machines. [substeps] If you are making badges to identify the wearers, you want a machine that produces nfsl.', 'If you want to display logos or clever sayings, you probably want a badge making machine geared to producing round buttons. [substeps] Some badge making machines are capable of creating magnets as well as badges or buttons.', 'Write \\\" badges & badges making machines \\\" onto the top of the badge. [title] Ask customers before throwing your badges out.', '[substeps] If you are making badges for working or education, then you may need a badge making machine to produce badges for people in need. [title] Decide if your machine is suitable for your area.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}