{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"520bf6f8-6f76-52aa-a2b7-ae50239d91d4","task_key":"default--validation--5507","task_revision_id":"1","upstream_id":"5507","short_description":"HellaSwag validation 5507","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"As they run, people are on the side of the road encouraging them with water on their tables, signs and clapping for them. Aside from the people, there are also large red blow ups on the side illustrating the hydration stations. finally\",\"endings\":\"[', the streaker finish on time with the wave of his hand and walks off.', ', a large green group of people on a dirt bike appears, cut, ready and they all are waiting for the talking with a ripple of dirt in the ground.', ', a young couple completes the marathon and a still image of them is shown at the end.', ', there is a blue screen with the caption lsuongs) walking the outside of a building and men holding bumper cars.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}