{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d75b0bf8-1e29-59ec-b4a2-323ab5e808db","task_key":"default--validation--24912","task_revision_id":"1","upstream_id":"24912","short_description":"HellaSwag validation 24912","config":"default","split":"validation","body":"{\"activity_label\":\"Doing crunches\",\"ctx\":\"There's a young girl dressed in a blue tank top and black tights demonstrating the proper way to do planks and crunches. she\",\"endings\":\"['then uses a long, flat bar to suture chopped wooden planks down each side.', 'attempts to do the cycles while exercise fully geared up and then we see her in inch long clothes.', \\\"'s demonstrating, in slow motion, how to assemble the proper drill for the jump.\\\", 'is laying on a blue yoga mat as she does crunches.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}