{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d11c165d-cb4a-545f-9372-73bebeb45c00","task_key":"default--validation--21720","task_revision_id":"1","upstream_id":"21720","short_description":"HellaSwag validation 21720","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to pressure flip [title] Have the right gear. [step] Ideally you should have a shortboard skateboard, although experts can do pressure flips on old school and cruiser boards. You won't be able to pull one off on a longboard.\",\"endings\":\"['[title] Open up the throttle and start turning. [step] Throttle pulley lifts gear about 4 to 6 degrees from you (this is recommended).', \\\"[substeps] Make sure you have a good amount of space to safely maneuver your board before you start to flip. Place the wrong gear on your flat side of the skateboard while your board's nose is facing upwards.\\\", \\\"Skate shoes aren't necessary for skating, but may be more comfortable and will last longer if you skate a lot. If you don't have skate shoes, wear something you won't mind damaging.\\\", 'A good rule of thumb is to have some extra gear in case you get lost on your own during a spin. Your back is most vulnerable at this point in the trick.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}