{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ccf7d628-34ad-5d0a-b010-bf97721093e8","task_key":"default--validation--39724","task_revision_id":"1","upstream_id":"39724","short_description":"HellaSwag validation 39724","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A kid is in the yard putting the ball on the side of the fence. An adult came and start to cut grass with the kid. the man\",\"endings\":\"['moved the goal net on the side and continue to cut the grass using the grass cutter, then he move the trampoline and move cut the grass as the boy is playing on the side.', 'pushed the kid and grabbed the grass.', 'went back and looked at the kid and take pictures of the girl.', 'walked up to the kid and started to take the leaves off the fence.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}