{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e05d9d8d-cce1-5b81-b41f-b8fe090157de","task_key":"default--validation--11242","task_revision_id":"1","upstream_id":"11242","short_description":"HellaSwag validation 11242","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to avoid being the subject of workplace gossip [title] Work hard. [step] If you do a great job and earn respect from your workplace peers, they will be less likely to gossip about you. Always pull your weight at work, and don't slack off.\",\"endings\":\"['Also, stay on top of the work. If you just end up missing an event, you will never be the topic of gossip.', 'Go the extra mile on projects and presentations. Pitch in to help your peers when they need assistance.', 'Even in situations where you are prone to gossiping, keep in mind you can maintain your standing. [substeps] Give praise when you do a good job and brag about your accomplishment when you succeed.', \\\"If you do all of your work, you will never be treated as the subject of gossip. Everyone's ways of doing things are different.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}