{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"991e936b-ec2a-50d1-9030-99adb35657c8","task_key":"default--validation--16134","task_revision_id":"1","upstream_id":"16134","short_description":"HellaSwag validation 16134","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to write business emails [title] Summarize the email in a 6-8 word subject. [step] It is important to make sure you include a subject, since an email that does not have a subject may be ignored. Your subject should be short and to the point and it should highlight the main message of the email.\",\"endings\":\"[\\\"[substeps] If you are writing to a business partner or a friend, try to summarize these first sentences in a 1-2 paragraph citation. The body of the citation should tell them just what you're sending and why.\\\", '[substeps] If your email is rhetorical, mention that on the subject line. [title] Include a signature at the bottom of the email and leave extra space at the bottom.', 'The ideal subject gives the reader all they need to know or informs them they need to make a decision. People who receive hundreds of emails a day may only open emails with particularly compelling subjects.', '[substeps] For example, if the subject of a business email is \\\" your tomato...\\\" then it should highlight it in the sender email field. If it is a marketing email, then it should highlight it in the sender email field.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}