{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c42decc4-9fad-50ce-bd33-7d461fd9fb1f","task_key":"default--validation--11950","task_revision_id":"1","upstream_id":"11950","short_description":"HellaSwag validation 11950","config":"default","split":"validation","body":"{\"activity_label\":\"Baking cookies\",\"ctx\":\"The woman mixes various ingredients together in a bowl and leads into her putting clumps of dough onto a pan. she\",\"endings\":\"['washes the dough with a sponge and then cuts it into bite size pieces.', 'slices the dough down the middle, then separates the pieces evenly, laying out the claymaking product.', 'puts the pan in the oven then takes it out to show off her chocolate ship cookies.', 'bowls the dough into balls and proceeds to hang them on the wall.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}