{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ca3e5a5c-db2c-5c96-bbf1-75e6ca005af7","task_key":"default--validation--5505","task_revision_id":"1","upstream_id":"5505","short_description":"HellaSwag validation 5505","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"Several individuals are dressed in workout clothes, tying up their shoes, and placing numbers on their shirts. All of the people meet at the starting line and begin a marathon throughout the city. as they\",\"endings\":\"['run, people are on the side of the road encouraging them with water on their tables, signs and clapping for them.', 'go, they are shown playing some stringed instruments.', 'finish, someone drops an item, along with their name, at the end of the video.', 'run, they pass two teams of players who have long lines and a large number of spectators watching or cheering them on.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}