{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ca1cca62-caae-5035-97b2-cb6c890caddf","task_key":"default--validation--16317","task_revision_id":"1","upstream_id":"16317","short_description":"HellaSwag validation 16317","config":"default","split":"validation","body":"{\"activity_label\":\"Playing water polo\",\"ctx\":\"Two teams play water volleyball in an outdoor pool with a small audience surrounding them including a referee dressed in white on the sidelines and digital scoreboard. the two team in the water\",\"endings\":\"['competitively pursue a yellow volleyball from the water.', 'demonstrate some sort of jump with rocks, the referee claps and a series of images of the contestants are shown.', 'won two of the games in a competitive trajectory.', 'batter at the net then hit the ball at the net and the referees clap with their hands and attempt once again the volley as if it were sport and this time the ball rolls under the net and hits it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}