{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bd8ac24e-bb89-59df-9f1d-1049fc945485","task_key":"default--validation--16620","task_revision_id":"1","upstream_id":"16620","short_description":"HellaSwag validation 16620","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"An old black and white clip plays and a tall man starts running towards a pole that is up high, and there are a lot of people in the general area doing their own thing or watching him. when the man\",\"endings\":\"['reaches the pole he jumps very high, clears the pole, lands on the matting below it and people cheer while he gets up and runs back to the area he started.', \\\"reaches his pole it comes down with such a little weight, it doesn't even break his fall.\\\", 'reaches the pole he looks at it and then continue to run towards it till he gets to shoulders height.', 'gets to the pole and says something, some of the people start jumping beside him while some of them go to other areas.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}