{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a16fea4d-79c4-522f-bc6c-7955f05a2349","task_key":"default--validation--18430","task_revision_id":"1","upstream_id":"18430","short_description":"HellaSwag validation 18430","config":"default","split":"validation","body":"{\"activity_label\":\"Getting a piercing\",\"ctx\":\"She then lays down on the table. She gets a clamp put under her eye pinching the skin. the piercer\",\"endings\":\"['opts to achean her.', 'cleans off the individual items of paint applied to the eye.', 'puts an earring in and she sits up.', 'goes down her eyelid then she puts it back on.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}