{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01516871-2e33-5461-8626-8e71d5c9edb9","task_key":"default--validation--29987","task_revision_id":"1","upstream_id":"29987","short_description":"HellaSwag validation 29987","config":"default","split":"validation","body":"{\"activity_label\":\"Making a cake\",\"ctx\":\"Three kids are at the table, the little girl at the center is mixing the bowl while dancing. The kids are licking their fingers and spatula. the girls\",\"endings\":\"['then grab their salt and put them on plates.', 'throw more spoonfuls of the pancake mix on to their part and stop licking their fingers and moved their forks and spoons.', 'places the bowls in a showcase.', 'are adding ingredients in the bowl and then mixing the ingredients at the same time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}