{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"f9fe8042-461a-5e4e-b5de-8d137596637e","task_key":"default--validation--1752","task_revision_id":"2","upstream_id":"1752","short_description":"HellaSwag validation 1752","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"A female basketball coach is seen instructing a group of girl basketball players who are standing in line on a basketball court. the first girl\",\"endings\":\"['walks across the court with the ball and keeps walking then pulling the girls to the other side of the court and the girls begin playing volleyball rhythmically rolling on the floor as the coach helps them follow how to properly do things.', 'line up and stand behind a dummy dummy.', 'trying to get the ball to go far past the basket and hit it back towards the basket while her coach continues teaching her.', 'passes to another coach and then runs to the net and takes a layup.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}