{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a41e7751-6e69-5a25-9127-d02360bbaa58","task_key":"default--validation--12510","task_revision_id":"1","upstream_id":"12510","short_description":"HellaSwag validation 12510","config":"default","split":"validation","body":"{\"activity_label\":\"Plastering\",\"ctx\":\"A close up of tools and objects are shown laid out leading into several clips of a mean measuring a wall and laying plaster down on it. the man\",\"endings\":\"['shows how to make plaster while transitioning with him making plaster and continues to show more clips of him laying it down.', 'manipulates the objects and applies plaster around the corners and vigorously pokes holes and mortar chips.', 'then begins carving the wall along with a large metal ring attached to it and showing a close up of another wall in the same exact spot.', 'uses a grout tool to plaster down and lays hits the plaster down as well as plaster out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}