{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"f6cab8ab-8506-5525-8744-e35a6adf5262","task_key":"default--validation--2301","task_revision_id":"2","upstream_id":"2301","short_description":"HellaSwag validation 2301","config":"default","split":"validation","body":"{\"activity_label\":\"Putting on makeup\",\"ctx\":\"A woman is seen putting makeup on her face and leads into several pictures of makeup. Many more pictures of products are shown with text across the screen. more pictures of products\",\"endings\":\"['are shown on the screen leads into a woman putting makeup on herself.', 'are shown and ends with the camera shining off into the distance and the woman walking away.', 'are shown and ends with a woman hula hooping with herself.', 'are shown that end with more text and a woman pushing away makeup.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}