{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e8f94c07-ebfb-5aa3-86a7-d37454a818ba","task_key":"default--validation--273","task_revision_id":"1","upstream_id":"273","short_description":"HellaSwag validation 273","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to prevent sweaters from stretching [title] Read the care label. [step] Always read the care label on the inside of your sweaters. You should find out if the sweaters can be washed at home or if they must be dry cleaned.\",\"endings\":\"[\\\"If you don't understand the care label, then look it up on the back of the packaging. [title] Look for straps that are too big.\\\", 'Find out what washing instructions are included with the care label. Make sure you read the label carefully so you can determine how many washes will be necessary for each garment.', 'If a sweater is currently dry cleaned but they have stretch marks on them, it might be an incorrect diagnosis. [substeps] Stop buying either new or used sweaters to wash them.', \\\"The care label will also tell you how to wash the sweaters, although you should plan on hand washing them just to be safe. [substeps] If your sweater has to be dry cleaned, don't even consider hand washing it at home.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}