{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6316e6d8-2ec6-5ed8-87a5-c3c0848a4756","task_key":"default--validation--18117","task_revision_id":"1","upstream_id":"18117","short_description":"HellaSwag validation 18117","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to oil your hair [title] Decide how many oils to use. [step] You can use just one oil on your hair, or two to three oils. It all depends on how much money you want to spend and how intensive you want your oil treatment to be.\",\"endings\":\"['[substeps] The two basic types of oils are carrier oils and essential oils. Carrier oils are used as a base into which the more concentrated essential oils are mixed.', 'The amount of oil you use can vary depending on the type of dyed hair you have. [substeps] Some products require 100 to 140 oil.', \\\"[substeps] Enough oil can become very stubborn, and you won't be able to keep it out much longer. Acne is highly likely to result in extensive hair damage and hairline acne.\\\", \\\"Therefore, you'll need to choose one oil for every 20 pounds you have to add. [substeps] Massage oil is great for coconut and olive hair.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}