{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41466ea5-487c-504a-ab6f-b52b7827cc7b","task_key":"default--validation--37385","task_revision_id":"1","upstream_id":"37385","short_description":"HellaSwag validation 37385","config":"default","split":"validation","body":"{\"activity_label\":\"Playing badminton\",\"ctx\":\"The players have a long volley until the play in the foreground misses and the birdie lands at her feet. the\",\"endings\":\"['start with a 170 the 2nd time and she almost misses the birdie.', 'players raise their arms in victory as the crowd gives a cheer.', 'game ensues and the goalie dives into the net and hits the wood to pull the birdie out of the net.', 'players volley and the birdie is hit out of view and the player retrieves it then serves.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}