{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"2f65e7f5-d7a9-5dec-ade7-e6f16d86fb7e","task_key":"default--validation--1115","task_revision_id":"2","upstream_id":"1115","short_description":"HellaSwag validation 1115","config":"default","split":"validation","body":"{\"activity_label\":\"Throwing darts\",\"ctx\":\"A man is seen throwing a dart onto a board and leads into a woman speaking. the man\",\"endings\":\"['tosses darts into the water while people measure the distance and run after them.', 'gives her several tips and she continues creating darts.', 'throws several more darts while taking the time to speak to the woman.', 'holds behind him and the woman continues speaking.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}