{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ee49a40b-5ce9-581e-8896-55d2fd7d1065","task_key":"default--validation--14248","task_revision_id":"1","upstream_id":"14248","short_description":"HellaSwag validation 14248","config":"default","split":"validation","body":"{\"activity_label\":\"Tennis serve with ball bouncing\",\"ctx\":\". a man dressed in a black sweatshirt and black shorts, along with another person dressed in white athletic wear\",\"endings\":\"[', wearing blue shorts, brown leotard and no shirt will come on stage and begin kicking the ball.', 'raises a blue barrel, sits down, grabs the barrel and begins to squeeze the stiff barrel hard and then lifts his hand casually up and down to flip the barrel, then lowers it like a discus and puts a lid on to keep the beer from getting out.', ', appears on the screen and then the legend appears again, speaking to a woman in a bedroom and the world begins to ends.', 'are standing in a tennis court near the net.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}