{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36eeebc3-6949-5695-80f0-63cce57e805a","task_key":"default--validation--28971","task_revision_id":"1","upstream_id":"28971","short_description":"HellaSwag validation 28971","config":"default","split":"validation","body":"{\"activity_label\":\"Playing badminton\",\"ctx\":\"Two men are seen playing a game of badminton with one another and leads into one of them speaking to the camera. the man\",\"endings\":\"['continues to speak to the camera while showing off a set of perfect shot penises and using a pencil to draw in designs.', 'continues to speak to the camera and leads into several clips of him speaking to the camera.', 'then demonstrates how to properly hit the birdee while moving in slow motion and leading back into him speaking and playing with others.', 'leads the other back and forth while the lead mans shakes his fists and ends by holding a white towel from behind him and speaking to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}