{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c9f0245f-bf94-5f90-893e-895d654fce55","task_key":"default--validation--11300","task_revision_id":"1","upstream_id":"11300","short_description":"HellaSwag validation 11300","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to swat a fly without a fly swatter [title] Slap the fly with one hand. [step] Swat a fly by trapping it between one hand and a hard, flat surface that you have gotten it to land on. Move slowly toward the fly, then slap it quickly and firmly with your hand.\",\"endings\":\"[\\\"A very light slap like this works well to send the fly flying, but even a smack would work very well depending on the fly's size. [substeps] Using a fly swatter should be enough to remove the fly fly from your hand.\\\", \\\"The movement of your slap will force it to land on the floor, suggesting that you are probably right for it. [substeps] If the fly lands on the ground instead of a tough surface, you've caught it mid-flight or might have hit it again.\\\", \\\"[substeps] Approach the fly very slowly until you are within an arm's length of it. Keep your arm still until you are in position, then move it as quickly as possible in one motion toward the fly to hit it.\\\", 'Repeat this until the fly lands quickly and you can swat any fly quickly. You may not get to swat very easily, but most people do.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}