{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb8c927d-93fd-5264-a6cc-5694dac3e570","task_key":"default--validation--15570","task_revision_id":"1","upstream_id":"15570","short_description":"HellaSwag validation 15570","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make loose green tea come out perfectly [title] Get your ingredients and prepare your equipment. [title] Ensure that everything is clean and ready for use. [title] Purify the water.\",\"endings\":\"['[step] Do this by pouring 10/220 cup (250 ml) water into a water bath and shaking it for 3 seconds. [substeps] If necessary add 10 drops of an essential oil to the water bath which will create a faint scent in the water.', '[step] Do this by using the microwave, setting the temperature to 375ºf (176ºc) and water. [title] Pour the dissolved sugar over the water.', '[step] Water is a key ingredient in making most tea because it helps create steam which helps to keep the temperature of the tea down. It also allows the liquid to flow evenly through the air.', \\\"[step] Tap water might have a taste that affects the quality of green tea by affecting it's aroma and flavor. However, you can ignore this step if you are not in a hard water area.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}