{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5fa05a8-67aa-5c28-a5b4-b7ed45fd572e","task_key":"default--validation--48908","task_revision_id":"1","upstream_id":"48908","short_description":"HellaSwag validation 48908","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write an accomplishment report [title] Open with a summary paragraph. [step] At the top of the, summarize the overview. Tell the readers the overall achievements.\",\"endings\":\"['[substeps] Perhaps you are writing an accomplishment report for a non-profit organization. You could summarize successes, such as the fact you organized events that benefit stakeholders, gained industry recognition, and created linkages with partners.', 'Include how each accomplishment helped you achieve your goal. Tell your reader any specific challenges or successes your company faced in the next 20 years.', '[substeps] For instance, you might write: \\\" i spent the rest of the day pulling an all nighter preparing for the test on thursday but saved ten minutes for an assignment on tuesday. I accidentally missed the main topic of my paper.', 'What things did they do well? What goals did they fulfill? Identify what parts of the project led up to her success. Aim to improve every step in her success.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}