{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dff1e31e-3b32-59a9-9268-d89c7ff2b1d5","task_key":"default--validation--7893","task_revision_id":"1","upstream_id":"7893","short_description":"HellaSwag validation 7893","config":"default","split":"validation","body":"{\"activity_label\":\"Baking cookies\",\"ctx\":\"The chef demonstrates how to mix the butter and flour in a blender to blend the dough. Then she knead the dough together to make a rectangular roll. she\",\"endings\":\"['sprinkles some sugar on a parchment paper and rolls the dough in it and covers it before keeping it in the refrigerator.', 'adds the mixes of other ingredients into the dough.', 'rolls the dough around and makes a sandwich.', 'measures flour and dough and knits it into a plastic bag.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}