{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8bac9af2-237e-527c-9cbd-323516503eaa","task_key":"default--validation--47202","task_revision_id":"1","upstream_id":"47202","short_description":"HellaSwag validation 47202","config":"default","split":"validation","body":"{\"activity_label\":\"Cleaning shoes\",\"ctx\":\"She starts by demonstrating a dry method to clean leather and then moves on to a wet method used to clean leather. she\",\"endings\":\"['demonstrates a bit more riding on a longboard and the video shows several horses getting harnessed.', 'continues using the wet technique and this time applying a different paint purpose.', 'proceeded to show several different products are best at properly cleaning the leather.', 'continues running the brush through the leather, thoroughly wiping sections of it and finally hair hying the part of the leather where she has wiped.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}