{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb7045a7-baf9-5e77-9744-12ad63ac61ad","task_key":"default--validation--43507","task_revision_id":"1","upstream_id":"43507","short_description":"HellaSwag validation 43507","config":"default","split":"validation","body":"{\"activity_label\":\"Playing ice hockey\",\"ctx\":\"As they continue to play, the referee begins to skate back and forth on the side in circles. after several starts and finishes\",\"endings\":\"[', a man in a red shirt skates through.', ', the games keeps going and the goalie is successful in blocking the puck each and every time.', ', the the team gives all their credits for the video.', ', the clock strikes and the players are congratulated for their hard work, along with many others.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}