{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83780f39-8f92-57c6-8fc1-c47f3875e7c2","task_key":"default--validation--11860","task_revision_id":"1","upstream_id":"11860","short_description":"HellaSwag validation 11860","config":"default","split":"validation","body":"{\"activity_label\":\"Elliptical trainer\",\"ctx\":\"The woman in red continues explaining the use of the elliptical. Then a demonstration of how the legs must move on the elliptical is shown. the woman in red\",\"endings\":\"['continues explaining the mechanics and features, but the lady in the red shirt gets distracted for a little bit.', 'appears, lying down on the elliptical and explains how the switch works and in the end ends with the key being shown dropped on the screen.', 'further explains benefits of the workout with an elliptical.', 'explains some more demonstrating how to move the legs as well as talking to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}