{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b374920-0ea9-5b45-ae77-e9f606cef330","task_key":"default--validation--11585","task_revision_id":"1","upstream_id":"11585","short_description":"HellaSwag validation 11585","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get rich using the power of your mind [title] Know what you desire. [step] What you want is the first thing you should be aware of. Listen to your desires and there must be purpose in your desire.\",\"endings\":\"['No matter how impossible it looks at the beginning, dream big. Your feelings should be attached with whatever you desire.', 'It may be something small, like killing bugs and putting wood chips on a fire. Let yourself be there but let your feelings get you.', 'If you are proud of what you want, then others in the world will be proud of you too! [title] Understand your personal values. [step] Redefine your values and see them in a more realistic way.', 'It should have something to do with the world as a whole. If it is for the purpose of something, the best thing you can do is to pick some material material that suits your needs and nature.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}