{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c96f3e93-8552-5699-bc98-1f8faa25bf20","task_key":"default--validation--15807","task_revision_id":"1","upstream_id":"15807","short_description":"HellaSwag validation 15807","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to pick earrings when you get your ears pierced [title] Get your earrings in the proper metal. [step] Surgical stainless steel earrings are typically your best bet for new piercings, as this metal is the least likely to cause an allergic reaction. Nickel and cobalt allergies are very common, so avoid these metals when you first get your ears pierced.\",\"endings\":\"['This can make it very time-consuming to get your ears pierced, even with safe equipment. Remember that the choice of metal is yours, and it is easier to pick from the easiest metals.', 'When you get your ears pierced, ask the piercer to help you pick out earrings so that you get the right metal. [substeps] Besides surgical stainless steel, other fairly safe options include platinum, titanium, and 14 k gold.', 'Most modern earrings can be made from titanium or silver, but titanium jewelry is very rarer. [substeps] Silver earrings are the most popular and usually come in various colors, such as gold or titanium.', 'Custom metals like nickel and cobalt have less than 80% sensitivity. [substeps] Specialty steel cutters are also also very popular for these kinds of metals.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}