{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c34fec3-f273-5cbd-b5af-d36fa328f2b7","task_key":"default--validation--22817","task_revision_id":"1","upstream_id":"22817","short_description":"HellaSwag validation 22817","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A man walks along a bare field while a woman walks around beside him. He walks back and fourth preparing to jump and eventually jumps over a pole. a girl\",\"endings\":\"['in a bra follows him around the field.', 'walks out of frame and transitions into the man while the woman walks around.', 'claps for him and the same move is shown again in slow motion.', 'use her hands to assist him as someone else helps.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}