{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5ea42ea-084b-573b-bfca-42c83ee7015e","task_key":"default--validation--30379","task_revision_id":"1","upstream_id":"30379","short_description":"HellaSwag validation 30379","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to stop an angora sweater from shedding [title] Determine if your sweater can be frozen. [step] This method of freezing works best for angora sweaters that are blended with another material. If your sweater is 100% angora, your sweater will still shed because of the high amount of angora.\",\"endings\":\"[\\\"[title] Look for a sweater made of wool or suede. [step] These fibers won't be able to cause the sweater to shrink.\\\", '[substeps] This method is particularly important to avoid in larger sizes of sweaters. It will completely melt the outer layer of the sweater.', \\\"[substeps] If your sweater is made from synthetic fibers, it is not ideal at all. Synthetic fibers will degrade over time, especially in colder weather, so don't use this method if you are planning to keep this sweater.\\\", 'The freezing method relies on making the fibers easier to shake out. So, after you freeze the sweater you can vigorously shake out all of the fibers that will fall out so they come out at once instead of throughout the day.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}