{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"90b01ce6-f7f6-5531-9a4d-a419660310c1","task_key":"default--validation--16472","task_revision_id":"1","upstream_id":"16472","short_description":"HellaSwag validation 16472","config":"default","split":"validation","body":"{\"activity_label\":\"Painting furniture\",\"ctx\":\"The man covers the entire chest from the very top to the bottom with the spray varnish which is coming from a long hose attached to a metal pump. the man\",\"endings\":\"['plays the music cutting around different alifes of the product.', 'uses the hose on his legs and hands taking off the hose from the hose that is attached to the pump.', 'finishes spraying the chest and hangs the nozzle up and walks away as the camera focuses on the finished product.', 'continues plastering the chest and the adapting metal pump and then grabs another container of the paint and begins to use it to paint a height mirror them and use it to scrub excess paint off of it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}