{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5be732f-a267-59d6-9beb-75b7a4ebdfa1","task_key":"default--validation--13004","task_revision_id":"1","upstream_id":"13004","short_description":"HellaSwag validation 13004","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use the inversion method to grow hair [title] Choose your oil. [step] There is not one agreed upon oil for the inversion method. Try olive oil, castor oil, coconut oil, grape seed oil, or moroccan argon oil, if you want to use natural ingredients.\",\"endings\":\"['In most countries, commercial oil will not be included in the treatments. [substeps] Choose an oil based on an herb called mint, chamomile oil, or chickweed oil.', '[substeps] Choose an oil with an aroma that you find soothing. Since the inversion method incorporates massage, pick a scented oil that you find relaxing.', \\\"[substeps] Adults need one/8 cup (55 grams) of oil for every ten pounds you're losing. If you're starting breastfeeding or are doing surgical work, make sure to think about oil preferences.\\\", 'However, you should definitely make sure to use other oils that are suitable for this experiment. [substeps] Avocado oil has been shown to thicken hair faster than olive oil because of the more concentrated oil used.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}