{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0d594c02-70b2-572f-8312-062611d7df53","task_key":"default--validation--10331","task_revision_id":"1","upstream_id":"10331","short_description":"HellaSwag validation 10331","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to communicate in volleyball [title] Call the ball when you know you're within range and can hit it. [step] Be sure to enunciate and project your voice so your teammates can hear you loud and clear. Not calling loudly enough is a common cause of failure for many plays and formations.\",\"endings\":\"['[title] Match your balance to your putter. [step] Your batter should aim for the point between 1.75 and 1.89 feet (1.8 and 1.1 m) from the body of the ball.', \\\"Calling too loud or sounding too sporadic can lead to mistakes. [title] Start the game when you're even with your team.\\\", '[title] Communicate before you hit to preserve time and avoid losing points for your hits. [title] Strike their ball off the baseline so your teammates can hit their own ball but not hit theirs.', '[substeps] Call your teammate off if you notice you both are trying to hit the ball. Something as simple as \\\" mine, \\\" or \\\" i got it \\\" should communicate your intent.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}