{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96bb73cb-9463-5b1b-9a71-9b3b70a4289b","task_key":"default--validation--3660","task_revision_id":"1","upstream_id":"3660","short_description":"HellaSwag validation 3660","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"A violinist, lindsay stirling is performing live with some other participants on a stage in front of an audience. She begins by playing the violin while the other performers dance in a choreographed fashion. the violinist\",\"endings\":\"['then pauses to speak to the audience.', 'dances as she plays the instrument along with five other dancers.', 'continues to play as couples dance in the center.', 'is seen playing together at some point while while several people are filming him.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}