{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"690f3275-ab61-5974-a880-99cf1236a267","task_key":"default--validation--16318","task_revision_id":"1","upstream_id":"16318","short_description":"HellaSwag validation 16318","config":"default","split":"validation","body":"{\"activity_label\":\"Playing water polo\",\"ctx\":\"Two teams play water volleyball in an outdoor pool with a small audience surrounding them including a referee dressed in white on the sidelines and digital scoreboard. The two team in the water competitively pursue a yellow volleyball from the water. the two teams\",\"endings\":\"['end up winning the competition by two points.', 'hit the ball and swim after the ball spanning the length of the pool as the referee at times interrupts with a whistle.', 'play a blue and yellow game in the indoor pool.', 'continue to play and receive occasional water strokes in the pool while the coach instructs on how to hold the ball, volley ball and all the returns.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}