{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c22890d-adf5-57f6-93d2-7064407919f8","task_key":"default--validation--27525","task_revision_id":"1","upstream_id":"27525","short_description":"HellaSwag validation 27525","config":"default","split":"validation","body":"{\"activity_label\":\"Making a cake\",\"ctx\":\"All three children wipe the baking tray and pour the batter into it. Then they stick it in the oven and bake the cake. once the cake\",\"endings\":\"[\\\"is done, they continue to talk and tease each other before wiping it down and stopping it's doing.\\\", 'is ready they remove the wrapping paper and show the complete cake.', 'is ready, they put some white butter forsting on and decorate the cake with sprinkles.', 'is served, they continue to bake with the cookies.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}