{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1596fa0e-e131-5914-918b-36535b428500","task_key":"default--validation--12597","task_revision_id":"1","upstream_id":"12597","short_description":"HellaSwag validation 12597","config":"default","split":"validation","body":"{\"activity_label\":\"Snow tubing\",\"ctx\":\"A person sits in an inter tube at the top of a hill then rides down the slope laughing. the family\",\"endings\":\"['rides along the top of the snowy mountains on a tube out of site of others.', 'then parts and all the neighborhood are on a slack line doing parallel jumps being pulled by the inter tube during his skiing vacations.', 'stops when they reach the bottom and press their nose mask in.', 'straps together the inter tubes in a circular grouping with the kids sitting in them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}