{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"10d0a4d9-4669-5819-b114-c3f50b32bc25","task_key":"default--validation--32250","task_revision_id":"1","upstream_id":"32250","short_description":"HellaSwag validation 32250","config":"default","split":"validation","body":"{\"activity_label\":\"BMX\",\"ctx\":\"When the gates go down they all go as fast as they can through the obstacle of dirt. There are a few people sitting by in the stands watching as the bikers go off. they\",\"endings\":\"['arrive to a dirt covered area where there are lots of tents and gear on.', 'even show some of the kids waiting and having a good time watching the race.', 'continue going as riders are parked on the trail waiting to ride the bikes.', 'ride the dust bikes down the track only stopping to rest before leaving.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}