{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1b95a9ce-e595-5476-a060-649d9bc84e96","task_key":"default--validation--2048--variant-0c83528790e9","task_revision_id":"1","upstream_id":"2048--variant-0c83528790e9","short_description":"HellaSwag validation 2048","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to care for gold [title] Remove your gold jewelry before showering, bathing, or cleaning. [step] The exposure to soap and chemicals can dull the gold, necessitating more frequent cleanings. It can also scratch or break down the gold, which is a soft metal and easily damaged.\",\"endings\":\"[\\\"[title] Don't wear your gold jewelry in the pool. [step] Chlorine, like other chemicals, can permanently discolor gold.\\\", \\\"[substeps] Frequent shower and cleaning can also slow down the aging process. Rubbing your jewelry in baby shampoo can easily dull and discolor the gold's shine.\\\", \\\"Take special caution with this precaution if you're a gold enthusiast. Drop or remove as much jewelry as possible before showering or bathing.\\\", 'Washing and bathing regularly is the ideal way to prevent this. [substeps] Read all the instructions that come with your gold jewelry before it is purchased to ensure that you comply with any particular care terms.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}