{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9038901-7d04-5889-81d5-ffc70d2f56f1","task_key":"default--validation--32441","task_revision_id":"1","upstream_id":"32441","short_description":"HellaSwag validation 32441","config":"default","split":"validation","body":"{\"activity_label\":\"BMX\",\"ctx\":\"A camera shows a person riding around a dirt track on a bike moving at a quick speed. several more shots\",\"endings\":\"['are shown of people riding the dirt track on bikes while the camera captures them from several angles.', 'are shown of people riding around on bikes and speaking to one another.', 'are shown of people riding in different speeds and going over hills passing where there is much higher grass.', 'are shown of people riding around on the dirt track as well as riding on a stationary bike.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}