{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce271599-3d34-5164-a496-8720d0b19f5a","task_key":"default--validation--41580","task_revision_id":"1","upstream_id":"41580","short_description":"HellaSwag validation 41580","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to tell how pregnant you are [title] Keep track of the date of your last menstrual cycle. [step] The first day of the last menstrual cycle is actually considered the first day of pregnancy. [substeps] Marking this date on your calendar will also help you determine the date of conception.\",\"endings\":\"['You can either have several cycles in a single day, or you can have multiple cycles in a single month or three cycles in a single day. Keep in mind that your first period is usually the first regular menstrual cycle.', 'If you begin your cycle later than this same date, your time is correct. If in doubt, verify that you are pregnant to make sure that you are pregnant (and not any sexually embarrassing or suspicious symptoms that you may present in the future).', \\\"If you have multiple periods each month, you'll also know when you first spawn. If you do not have a menstrual cycle, it is possible you are pregnant.\\\", 'Fertilization is most likely to occur about 2 weeks after your last menstrual cycle. You are unlikely to experience symptoms until about 2 to 3 weeks after week 1.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}