{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9537e27d-9183-5178-b3fc-aee421ac587f","task_key":"default--validation--8975","task_revision_id":"1","upstream_id":"8975","short_description":"HellaSwag validation 8975","config":"default","split":"validation","body":"{\"activity_label\":\"Removing curlers\",\"ctx\":\"This woman is showing viewers how to get big bouncy curls using hot rollers. first she\",\"endings\":\"['takes a test, she shakes them to see how much they will like the curls.', 'applies her hair directly to the nape of her neck to away from the rollers.', 'pours liquid over her hair and pushes so hard it causes a pink head and the hair sticks then curls back in again.', 'lets the rollers heat up and she puts them onto her hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}