{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa15d64b-77f3-5700-afed-0c2adb4b5576","task_key":"default--validation--30382","task_revision_id":"1","upstream_id":"30382","short_description":"HellaSwag validation 30382","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to flirt with a cancer [title] Let them lead. [step] Cancers are both born leaders and astute financial planners. If they are a part of a group, they are likely to be at the head, leading it.\",\"endings\":\"[\\\"They are also very good with their money and can be quite successful at anything they set out to achieve. [substeps] Don't be intimidated by this aspect of cancer.\\\", 'Walk up and ask them to let you join. The more likely they are to say yes, the more likely they are to give you a good chance at getting out.', \\\"Instead, they're often in charge of planning proposals. When a cancer research project's worth of information is revealed, they immediately intervene in your cancer plan.\\\", 'Watch the approach of your cancer woman closely. If your cancer woman seems nervous, or very insecure in any of her leadership roles, that should not indicate the woman is a serious wreck.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}