{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"514b75d4-2b2a-57b7-a231-2aaa9e304801","task_key":"default--validation--19800","task_revision_id":"1","upstream_id":"19800","short_description":"HellaSwag validation 19800","config":"default","split":"validation","body":"{\"activity_label\":\"Playing beach volleyball\",\"ctx\":\"Two people are seen walking to one another leads into them playing a volleyball match with others. The man continues taking over by pushing his team mate down and jumping on her back. he\",\"endings\":\"['leads them into another game where the people hit the ball all over each other and continue playing with one another.', 'swings her around and ends by throwing her down on the ground and laughing.', 'then takes a path around to attack his opponent and others surround him and treat them very well.', 'spikes the ball in the end and cheers with the woman.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}