{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c7b06ac5-2448-54f7-9d19-e079a7bcb70e","task_key":"default--validation--39501","task_revision_id":"1","upstream_id":"39501","short_description":"HellaSwag validation 39501","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to use teak oil on kitchen cabinets [title] Know what ingredients teak oil products contain. [step] First of all, there is very little oil in any of the commercial grade oil finishes. They are made up of a little oil mixed with mineral spirits and varnish.\",\"endings\":\"['Poe oil is a little like coconut oil but in a very strong form which gives this oil a very pleasant flavor. Compare oils from all the commercial grade oils and see what each form smells like.', \\\"Essential oils should be used sparingly and then applied directly to the cabinet's caulk board with a brush. Second, there are at least a few essential oils that require a little extra oil to get rid of their grease and smudge.\\\", \\\"Here's where : [substeps] Whole deglaze is an old oil used for stainless steel, and oils are not waterproof or it becomes lumpy because of the lack of oil in its oil storage. If you want full coverage, opt for a gold or ivory teak oil.\\\", '[substeps] The lingo in the cabinetmaking trade is \\\" wiping varnish \\\" due to the primary content. You can define it as thinned out varnish with a little teak oil mixed in.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}