{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6466630b-a5dc-5d32-a130-ee243b331e73","task_key":"default--validation--10559","task_revision_id":"1","upstream_id":"10559","short_description":"HellaSwag validation 10559","config":"default","split":"validation","body":"{\"activity_label\":\"Fixing bicycle\",\"ctx\":\"Then he unscrews the base to level it out to his liking. Then he sits on the bike seat to achieve the best results. then he\",\"endings\":\"['is shown trying to model the seat with a different size of handle.', 'prepares an exercise equipment and goes front of the bike so he can talk to the camera.', 'adjusts the handle bars again and makes sure they are firmly in place.', 'stands up on his pedals, speeds off, and lands with his feet on the ground.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}