{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a63ae95f-fb72-5d0a-8c3d-c37e285c195f","task_key":"default--validation--34074","task_revision_id":"1","upstream_id":"34074","short_description":"HellaSwag validation 34074","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"There's a basketball player demonstrating various types of basketball hoop shooting techniques on behalf of world of hoops. he\",\"endings\":\"['is practicing what to do when a black ball returns and the aim changes.', 'is demonstrating how to use his perfected yellow balls on different occasions.', 'shows the different defenders and their strategies to improve the position of the shot-put.', 'begins by showing the basic technique of the regular lay-up throw, he shoots the ball directly into the hoop.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}