{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"71e70a39-dc5c-5c41-bba6-7a397e8e5a09","task_key":"default--validation--16825","task_revision_id":"1","upstream_id":"16825","short_description":"HellaSwag validation 16825","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to court a man [title] Make your intentions clear. [step] Courtship usually leads to marriage; it is a serious, forward-looking way to determine if you could spend your life with someone. Make sure that your man understands your intentions and what courtship means to you.\",\"endings\":\"['[substeps] Establishing clear expectations of the situations under consideration is the key element to successful courtship. Demand that every suitor make his or her advances free of emotions.', 'Simply explain your intentions clearly and then thank your partner for listening. Your husband should understand but do not feel pressured into doing something he has no say in.', \\\"[substeps] Tell him that courtship is something you take seriously because it's a great way to spend time together while still respecting your religious values. It's also a good way to build a solid foundation for a relationship.\\\", \\\"[substeps] Some courtship may be based on a grandiose goal: marriage, or adultery. Another may be based on a desire to end someone else's or establish a long-term relationship.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}