{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"349fac8e-c084-5572-a443-e9fd610cc5f3","task_key":"default--validation--11933","task_revision_id":"1","upstream_id":"11933","short_description":"HellaSwag validation 11933","config":"default","split":"validation","body":"{\"activity_label\":\"Drinking coffee\",\"ctx\":\"It is then shown that there are benefits and that one should moderate their intake to keep a balance. a website\",\"endings\":\"['informed of the difference difference is shown and it featured several clips of shopping lists on sitcom hosts.', 'is then shown to get more information on this topic.', 'listed number of freedoms where you can purchase a vehicle is shown.', 'shows up with a picture of a man and a woman that are close up a better picture of a yogurt cup.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}