{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"023cf578-01c6-5b0f-be09-ca6adb02c27b","task_key":"default--validation--17749","task_revision_id":"1","upstream_id":"17749","short_description":"HellaSwag validation 17749","config":"default","split":"validation","body":"{\"activity_label\":\"Trimming branches or hedges\",\"ctx\":\"A man is standing outside turning on his hedge trimmer turning it around to make the blades go. He then walks over to the hedges and begins to cut down the leaves on it. as he\",\"endings\":\"['cuts it asia walks towards the lawn.', 'is cutting by himself, a car drives by that is parked over in the yard.', 'is cutting it, he begins to walk into the hedges to cut the hedge evenly and shaking the cut leaves off with his hand.', 'is doing his mowing he trips and falls down.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}