{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"683695fd-b5d7-52e7-add8-1d9e51739da9","task_key":"default--validation--7310","task_revision_id":"1","upstream_id":"7310","short_description":"HellaSwag validation 7310","config":"default","split":"validation","body":"{\"activity_label\":\"Hand car wash\",\"ctx\":\"A large white truck with its hood open backs up. Marc man speaks to the camera by a vintage red car. he\",\"endings\":\"[\\\"wipes some car cleaner on the driver's side door and polishes it.\\\", 'proceeds to wipe down the shiny black car with sponge.', 'is painting the inside windshield purple.', \\\"vacuums back onto his and someone else's hose.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}