{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e4c3137c-1391-522d-b6fe-96b960555177","task_key":"default--validation--16352","task_revision_id":"1","upstream_id":"16352","short_description":"HellaSwag validation 16352","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"A close up of hashmarks is shown on a live, outdoor court, as a finger is seen from the side of the camera pointing to the marks on the court. a player\",\"endings\":\"['wearing sunglasses does a soccer throw over the net.', 'runs back and forth, quickly while continuing his run with others as well as slows to a stop and gives a thumbs up.', \\\"throws an orange ball, followed by another across the court and the camera eventually zooming in to show a person scoring an opponent, but she's apparently not very successful at the game, being picked up and played over by the game's behind gos, situals and male fans.\\\", 'is seen on the court demonstrating different layup and basketball moves with lettered templates accompanying the move to explain what they are.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}