{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6fd01c56-ff17-5986-b90f-8a7ea2ce81cc","task_key":"default--validation--5816--variant-ec4b863d9798","task_revision_id":"1","upstream_id":"5816--variant-ec4b863d9798","short_description":"HellaSwag validation 5816","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to slide in softball [title] Sprint at full speed to the next base. [step] Learn this slide before you try to do a more complicated slide. [substeps] Don't be afraid to get dirty.\",\"endings\":\"[\\\"Wear pants so that you don't injure your legs. Consider wearing sliding pants or sliding shorts underneath your pants if you're worried about being injured.\\\", \\\"Don't look back so long as you are landing softly on your toes! [title] Use the power plant as your dominant leg. [substeps] The power plant, or stance, allows you to bend your leg and land on your toes.\\\", 'The finish will vary from person to person, so be sure to familiarize yourself with the game beforehand to help you recall a slide. [title] Slide the ball in front of you on the plate.', 'If you do get dirty, you are picking a fight and taking your team to pieces. Always remember to keep your head up and run as you slide.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}