{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1fdfee6c-5f32-53d5-a7e0-f68cc2642d79","task_key":"default--validation--3055","task_revision_id":"1","upstream_id":"3055","short_description":"HellaSwag validation 3055","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to run a fantasy baseball league [title] Find people interested in playing fantasy baseball. [step] Form a league of between 8-20 teams. These may either be friends or anonymous individuals that you meet online.\",\"endings\":\"['You can line them up directly with you on the web. No one ever knows them, and all these people can do are only support you with the game, not give you advice, but genuinely want to see you succeed.', '[title] Choose a website to host the fantasy baseball league. [step] This is the easiest way to run a fantasy baseball league.', '[title] Develop teams that you feel want play. [step] Schedule regular practice sessions and encourage friendly competition.', 'A good initial search will help to find out if the person you want to play fantasy baseball with is a good fit for your team. [substeps] You should not attach real people to your team.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}