{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d9e9285-cee3-5e84-8731-e88182d9bfbd","task_key":"default--validation--6627","task_revision_id":"1","upstream_id":"6627","short_description":"HellaSwag validation 6627","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A view of the grass is shown, then a close up of a hand, and finally a man is shown riding on a riding lawn mower in a very large grassy area surrounded by a lot of trees. the man\",\"endings\":\"['continues to surf and seems to have a great self control as he continues to do various jumps and tricks along the grass.', 'begins mowing the large green grass with the other camera man following close behind.', 'waves to the camera, gives a peace sign, then drinks from a can.', 'is also shown riding around with a small yellow and white dog doing what appears to be a form of exercise.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}