{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"169cd790-2ab9-57ef-9fef-a6faf7abf205","task_key":"default--validation--34129","task_revision_id":"1","upstream_id":"34129","short_description":"HellaSwag validation 34129","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do a hair mask for oily hair [title] Brew tea. [step] Bring two cups of water to a boil. Pour in two tablespoons of nettle leaf.\",\"endings\":\"['Add chamomile flowers for light hair, or rosemary leaf for dark hair. Allow the mixture to cool, then pour the tea through a strainer to remove the herbs.', 'Allow the tea to steep for 10 minutes. [title] Make a mask with dandelion leaves.', 'Leave the tea to steep for a few minutes (three minutes if your hair is very thick), then strain the solution and boil the tea for about 30 seconds. Add honey and allow the tea to steep for a few minutes longer, then strain the tea and gently massage it onto your hair.', 'Let this tea steep for half an hour and turn down the heat. In a little while, your scalp should be flushed and your hair is ready for the mask.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}