{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a4bb10e-225a-5aa5-b9e7-1a373ecdb0f5","task_key":"default--validation--37386","task_revision_id":"1","upstream_id":"37386","short_description":"HellaSwag validation 37386","config":"default","split":"validation","body":"{\"activity_label\":\"Playing badminton\",\"ctx\":\"The players have a long volley until the play in the foreground misses and the birdie lands at her feet. The players volley and the birdie is hit out of view and the player retrieves it then serves. the\",\"endings\":\"['players volley while another man serves and the ball is dealt in the foreground.', 'players volley and the play is pressed to the back of the court hitting long shots.', 'game is over and the ref hits the ball.', 'screen goes back to its original color.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}