{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d427d65e-0e85-53c1-b0f5-a9e059123377","task_key":"default--validation--24475","task_revision_id":"1","upstream_id":"24475","short_description":"HellaSwag validation 24475","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to behave yourself [title] Be mindful of your noise level at all times. [step] Controlling your noise level is the biggest part of behaving. If you are having trouble maintaining an appropriate volume, stop and take a deep breath when you feel yourself getting louder.\",\"endings\":\"['Tell a friend to keep quiet or yell when you need help. Create a place in your house where you can be alone that cannot be disturbed by anyone.', 'Try to avoid loud conversations, where you may feel like the other person is strict about who you talk to. Try to be more observant and take cues from your surroundings when you find yourself grumbling.', 'It is hard to be articulate when you feel you are being controlled by your surroundings. [substeps] For example, you might start shouting things like \\\" stay away! \\\" and \\\" i\\\\'m going away.', 'Collect your thoughts and convey what you want to say in a respectful and reserved manner. Pay attention to your noise level and you can better control it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}