{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c883d25c-ba4b-581a-82fc-a2681f53dff9","task_key":"default--validation--11861","task_revision_id":"1","upstream_id":"11861","short_description":"HellaSwag validation 11861","config":"default","split":"validation","body":"{\"activity_label\":\"Elliptical trainer\",\"ctx\":\"Then a demonstration of how the legs must move on the elliptical is shown. The woman in red further explains benefits of the workout with an elliptical. the woman on the machine\",\"endings\":\"['moves back and fourth while the machine explains how to removed the elliptical section.', 'continues to show the viewers the correct way to exercising on the elliptical.', 'continues to talk about the difference between using the exercise machine and using a friend.', 'now begins to communicate with others and then still more women are shown engaging in female workout.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}