{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"489ef531-4863-5d0a-8554-2d5eff6e2187","task_key":"default--validation--10740","task_revision_id":"1","upstream_id":"10740","short_description":"HellaSwag validation 10740","config":"default","split":"validation","body":"{\"activity_label\":\"Calf roping\",\"ctx\":\"The focus switches to a man talking whose name is trevor brazile and it flashes from him talking and then to clips of him riding his horse in different arenas as he is also wrangling a small calf. another man by the name of stran smith\",\"endings\":\"[', the owner of the surfing is talking to the reporters about the event while demonstrating the swim jump technique he has learned from serving on numerous sporting events.', 'balaclavas is shown serving a white tennis ball.', 'is now talking and clips of him riding his horse in different arenas as he wrangles a calf also plays.', 'although his name is jeff schmidt, and then again with him is zack zuff.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}