{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"46fc9bb5-c969-5c06-bb09-b7773acce2db","task_key":"default--validation--7895","task_revision_id":"1","upstream_id":"7895","short_description":"HellaSwag validation 7895","config":"default","split":"validation","body":"{\"activity_label\":\"Baking cookies\",\"ctx\":\"She sprinkles some sugar on a parchment paper and rolls the dough in it and covers it before keeping it in the refrigerator. After cooling the dough, she takes it out and cuts them into rectangular pieces. she\",\"endings\":\"['places the raw cookie on a cookie sheet and bakes them.', 'seeds a cookie inside the cookie then puts them inside a rolled up package.', 'folds them in shape and makes the dough into a sling and a bowl with frosting.', 'removes the tube without cutting the dough.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}