{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c14e8fab-c6e0-510f-b29a-24f5a1bceec7","task_key":"default--validation--11123","task_revision_id":"1","upstream_id":"11123","short_description":"HellaSwag validation 11123","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"Several people are outside with running equipment and numbers pinned to them jogging at a slow pace at what seams to be a marathon or a 5k. a lady dressed in all black\",\"endings\":\"['sits on a bike and teaches a child going hurga run while some other ladies, a woman wearing an even black suit talks and then they continue to run.', 'begins to exercise in class full speed removing the ballerinas from her body.', 'however, seems to be the focal point throughout the video and she is walking at an extremely slow pace for the whole time.', 'pumps and displaying to the camera two little black turns around 5\\\\'10 \\\" and begins jogging down the course.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}