{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b388ee36-fde7-5c38-9950-e3cc3b93890e","task_key":"default--validation--7354","task_revision_id":"1","upstream_id":"7354","short_description":"HellaSwag validation 7354","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to know if you are ready to get engaged [title] Make sure you're in love. [step] If you are considering getting engaged, you probably love your partner. But are you in love with them? There actually is a difference, and it is important.\",\"endings\":\"['Know what you feel for your partner and vice versa and find a way to communicate. [substeps] It can be romantic to fall head over heels for someone.', '[substeps] Since you have been dating for a long time, you should have thought deeply about what you want out of your partner. Do not make hasty romantic decisions-you may end up breaking up with them eventually and this is when your doubts appear.', '[substeps] There are several differences between being in love and loving someone. When you are in love, you likely want to be physically intimate with your partner.', \\\"If you really love each other, you love your partner. [title] Remember that you don't have to use alcohol to get engaged.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}