{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6230124-a537-54ed-ba2e-f4be8f8118c1","task_key":"default--validation--12080","task_revision_id":"1","upstream_id":"12080","short_description":"HellaSwag validation 12080","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A weightlifter lifts over three bar bells above his head squatting under the weight as other gym attendees look on as he hoists the weight above his head and then allows the bell to fall to the floor. the man\",\"endings\":\"['gets up and squats as a crowd cheers him on and his large frame begins to spin around with very little exercise equipment on and he begins lifting more weights in the middle of the room.', 'performs several more lifts of the bell bending at the knees first and then hoisting the bell to the shoulders before throwing the weight on the floor.', 'then rolls out the excess weight over his shoulder portion and let it hit the ground before jogging across the gym in a sprint.', 'smiles as he lowers the weight down by putting his hands on the bar to steady himself.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}