{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"613c6f0a-3079-5469-ae4e-f5942a728e32","task_key":"default--validation--42054","task_revision_id":"1","upstream_id":"42054","short_description":"HellaSwag validation 42054","config":"default","split":"validation","body":"{\"activity_label\":\"Mixing drinks\",\"ctx\":\"He places the ingredients of alcohol and juice on a table, added the ingredients, as well as grenadine to a tall glass with ice. he\",\"endings\":\"['pours the drink into the glass, adds lime and people take their turns to drink it.', 'then pours all the alcohol and juice in it at the same time and places a small amount of chocolate syrup along the sides of the salad.', 'then pours liquors into the glass and shakes it, then adds a fresh drink.', 'displays the beautiful drink on the table.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}