{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40aadf2c-3a6b-5743-8db5-8f004df238e7","task_key":"default--validation--180--variant-0c89956129ca","task_revision_id":"1","upstream_id":"180--variant-0c89956129ca","short_description":"HellaSwag validation 180","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become a sports announcer [title] Research the field. [step] Before you pursue any formal education, do enough research to know what you can expect from this career field. Check into your expected duties, work environment, expected pay, and job outlook.\",\"endings\":\"['Keep a running log of the projected jobs and positions you plan to apply for. Check at your recruitment agency for more information.', '[substeps] Most sports announcers are either play-by-play announcers, talk show hosts, or sports anchors. Your duties may require you to announce plays, provide commentary, and interview guests.', \\\"Avoid going into a job bureau job applying for a position you have no past experience with. Most schools will offer career-related job regimes for those who haven't gone to college; unfortunately, online courses can be incredibly high the last few years.\\\", \\\"This will be especially useful in all types of sports sports. [title] Prepare to show the world you're capable of performing one day.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}