{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ee9be78-2a12-54bf-adbb-6b853a85fcbc","task_key":"default--validation--31760","task_revision_id":"1","upstream_id":"31760","short_description":"HellaSwag validation 31760","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to act perky [title] Seek out positive people. [step] If you want to be perky, a positive attitude is key. Seek to surround yourself with positive, happy people.\",\"endings\":\"['Consider making a list of positive people in your life. Chances are, you should include all of the people who you feel and know you love.', 'On the contrary, individuals who surround themselves with negative, unencouraging people tend to decrease their likelihood of resembling perky. Positive people go to every positive event and do what they can to make life better.', \\\"You'll end up feeling happier and, as a result, acting perkier. [substeps] Make plans with people who have positive attitudes.\\\", 'Having positive people around means having a social network. People who are positive are optimistic, affectionate, and generous.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}