{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a50a3b4e-a2e4-5d7a-bb6d-0d500c3eb39d","task_key":"default--validation--46641","task_revision_id":"1","upstream_id":"46641","short_description":"HellaSwag validation 46641","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to prepare for eid ul adha [title] Buy the required sacrificial animal 1-2 months in advance. [step] Doing things in advance saves you so much time. Buy the animal from a verified bazaar or shop or seller.\",\"endings\":\"['Check for the necessities in the animal as per the islamic shariah. Ask an islamic scholar for help if you are unsure of the type of animal to buy for sacrifice, or read the quran for further information.', \\\"[title] Ensure that you do not acquire a hunting license if you do not have a clear understanding of the breed. [step] Buy a hunter's license or some health-safe business license to protect yourself while hunting eid ul adha, and keep yourself in compliance and peace.\\\", \\\"[title] Buy the farm when the time comes. [step] The first 4 months in only a 1 month period are usually time spent at a bazaar, farmer's market, barn or farm or farm work.\\\", '[substeps] Many restaurants offer this phase of the process, so only buy from vendors in order to save money. If you have any questions about why your vegetarian bread is not vegetarian, ask around.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}