{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a81a511-6770-5b0d-a593-39e66209d7b7","task_key":"default--validation--39372","task_revision_id":"1","upstream_id":"39372","short_description":"HellaSwag validation 39372","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"The woman then applies water to her face, then pours contents from the little bottle onto her hands and she begins to rub it all over her face using her fingers. the woman\",\"endings\":\"['then shows a blow dryer with a rod attached and she grabs a towel and starts drying her hair.', 'spreads the water all over her face and then she uses a toothbrush for the dry skin.', 'then departs the bathroom and opens her kitchen cabinet to take out some salad dressing and in the process adds some strawberries she has in her hands which she spreads yes and then pours on a pancake.', 'speaks and then rinses her face off and then dries her face with a towel.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}