{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eba080c1-fb23-5f86-9c08-1fbc2228bcef","task_key":"default--validation--39741","task_revision_id":"1","upstream_id":"39741","short_description":"HellaSwag validation 39741","config":"default","split":"validation","body":"{\"activity_label\":\"Preparing salad\",\"ctx\":\"An elderly woman is shown talking before joining a group chopping lettuce at a table. The crowd holds out a green tarp as the lettuce and salad is poured into it. they\",\"endings\":\"['are then gathered in a grassy area to cut the lettuce.', 'cut the lettuce back and fourth from various chunks and continue talking with the woman.', 'then line up as bowls of the largest salad is served.', 'grind the lettuce off and are shown adding another one to the salad.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}