{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce9a442a-db21-5cbb-9071-f0095407ed11","task_key":"default--validation--12587","task_revision_id":"1","upstream_id":"12587","short_description":"HellaSwag validation 12587","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"The older girl is helping and teaching her younger sister to properly hold the violin by changing her head position. Then the older sister grabs her violin and tunes the strings of the violin. then older sister\",\"endings\":\"['resumes playing the violin for laugh.', 'begins playing the violin while the younger sister watches her.', 'showing with the second attempt to hold the bow and with the second arrow.', 'takes the violin back and that girl plays until her dad comes over to show her the proper pro-soprano positions to play the violin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}