{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"027d924b-fe5e-5be3-9fcf-07cba27e0680","task_key":"default--validation--31854","task_revision_id":"1","upstream_id":"31854","short_description":"HellaSwag validation 31854","config":"default","split":"validation","body":"{\"activity_label\":\"Ping-pong\",\"ctx\":\"The shirtless player returns the serve and the boy with the t-shirt misses the ball and has to pick it up and throw it to the other player for a re-serve. the two\",\"endings\":\"['men celebrate and slap each other on the back.', 'begin playing again while the boy in the middle on the sidelines observes the game.', 'teammates are now scoring impressive fils through the net, scoring numerous serves.', 'men hug and the shirtless player talks to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}