{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f6037bb9-749f-5cbf-98af-615738a50d1f","task_key":"default--validation--25887","task_revision_id":"1","upstream_id":"25887","short_description":"HellaSwag validation 25887","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to drive a golf cart [title] Understand golf carts. [step] Golf carts are classified under neighborhood electric vehicles. Despite the vehicle's name, golf carts are used off golf courses for a variety of reasons.\",\"endings\":\"['In example, golf carts are used because of their sleek, punctual appearance, and portability. [substeps] Certain carts sit on top of one another and may not function so well in busy communities.', 'It is the function of the chain that forces the club to move in order to roll up the ball. Essentially, golf carts are designed for the purpose in which they are used.', 'In order to stay in tip-top driving conditions, golf carts should be operated with a manual driver and with other drivers. These carts can make certain types of drivers feel uncomfortable or uninterested in doing something.', 'Golf carts fall between 15-35 mph top speed by regulation standards. You must be 14 years of age or older to drive a golf cart.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}