{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb8cc5f6-0213-5f76-b54f-3aa47d3ac0b9","task_key":"default--validation--33226","task_revision_id":"1","upstream_id":"33226","short_description":"HellaSwag validation 33226","config":"default","split":"validation","body":"{\"activity_label\":\"Having an ice cream\",\"ctx\":\"A little girl sits on front a cup of ice cream, then she takes a piece of food from the cup and eats it. then\",\"endings\":\"[', the girl takes a large stick and holds the stick to the edge of a pumpkin, then place the stick on top of the board and then play ice cream.', ', the girl finishes the ice cream and drinking the cup of ice cream she threw to the ground.', ', the little girl goes outside and eat apples close to the table, then she grabs nail tools and starts clipping the nails of a baby sitting.', ', the girl grab the spoon and eats the ice cream.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}