{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3665f8b4-cbad-5dff-af0e-5504d29406b7","task_key":"default--validation--39715","task_revision_id":"1","upstream_id":"39715","short_description":"HellaSwag validation 39715","config":"default","split":"validation","body":"{\"activity_label\":\"Playing badminton\",\"ctx\":\"A man and woman are shown standing on a tennis court passing a ball back and fourth. behind them\",\"endings\":\"[\\\"the man sharpens his knees as he takes hold of the ball missing all times and the woman's plays foosball.\\\", 'are another group of people playing and the man and woman continuously pass the ball back ad fourth to one another.', 'is a woman on a lawn and doing some tricks with the ball as well as a man in a room playing with the ball.', ', the man is blowing the whistle and the woman is talking.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}