{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"016aafa1-4db5-5835-b1be-eb9dc5d58e4e","task_key":"default--validation--45226","task_revision_id":"1","upstream_id":"45226","short_description":"HellaSwag validation 45226","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A man practices a long jump inside of an indoor gym using a interior sandbox for landing and a drawn race track isle. another man\",\"endings\":\"['practices hurling himself backward over a pole onto a gym mat inside of the gym.', \\\"presents it but it isn't a race and he starts a slow one.\\\", 'is shown practicing long jump over obstacles and metal gears while another sets aside to perform higher jumps in preparation.', 'runs and throws himself into the sand pit to land upright in a long jump.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}