{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d5a7189-91dd-5c6c-95b8-5d7ea0e4e1ca","task_key":"default--validation--24564","task_revision_id":"1","upstream_id":"24564","short_description":"HellaSwag validation 24564","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to get what you want for christmas [title] Think about your family and friends. [step] They probably want to make all your christmas wishes come true, but, many wishes come with a cost and it is usually money. Try hard not to ask for things that your family and friends can't afford.\",\"endings\":\"['[title] Really think about how much you really need for christmas. [step] Not too much for your occasion but still enough to satisfy your needs.', 'Consider asking for things that are only one or two of your family\\\\'s very favorite holiday traditions. This could be a great way to have this special \\\" family day \\\" together, but be aware that only those who let your family get perks can get those super budget \\\" christmas day days \\\".', \\\"It's ultimately up to you what you really want, but for everyone--who wants that? [title] Try to realize that you are not asking for something that is out of the box. [step] Wishing for something that is advertised, commercials, etc.\\\", 'If you have a lot of brothers and sisters it might be hard for your parent/s to get all of you everything you want. Remember there are other things parents have to pay for at christmas like food and decorations.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}