{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8875d542-5646-5784-a72f-e6344c0c9bd0","task_key":"default--validation--4091","task_revision_id":"1","upstream_id":"4091","short_description":"HellaSwag validation 4091","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"He moves closer to the camera so that you can see the detail of what he is doing with his hands. He begins to play different notes on the violin as he moves his fingers. the video\",\"endings\":\"['then zooms out and you can see him making notes on another piece of equipment.', 'transitions into him laying in bed as he plays.', 'ends with the closing credits shown across the screen.', 'stops going back to him playing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}