{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"053b67c9-75b6-5e04-bfd1-a1a77a503f4e","task_key":"default--validation--46899","task_revision_id":"1","upstream_id":"46899","short_description":"HellaSwag validation 46899","config":"default","split":"validation","body":"{\"activity_label\":\"Having an ice cream\",\"ctx\":\"A man dressed as aladdin is standing outside in his shop on the side of the road serving ice cream. a young girl\",\"endings\":\"['have a bowl is on her tray with two spoons sitting in it gray while the young man is helping her eat.', 'walks by on the side of the road wondering if the aladdin is doing something unusual, then he appears to come through the foliage to some fruits.', 'is standing on the side of the booth eating her ice cream as the man waits for his ice cream.', 'appears with a dog in her lap and she joins the man to exchange ice cream.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}