{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb1c948d-7cf3-55d3-a300-49e0be4e4d38","task_key":"default--validation--31885","task_revision_id":"1","upstream_id":"31885","short_description":"HellaSwag validation 31885","config":"default","split":"validation","body":"{\"activity_label\":\"Plataform diving\",\"ctx\":\"While diving he accidentally bumps his head on the diving board and injures himself. He expresses pain as he gets off the pool and his coach comes to help him. the second time\",\"endings\":\"['he gets in the pool the person instructs him on how to swim.', 'he lands on the diving board he is shocked and is the first to recover.', 'when the diver dives, he manages to successfully make it into the swimming pool without injuries.', 'that he fails he walks to catch his breaths.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}