{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8eb19bc2-0cf8-5fad-a424-bfd65c9d8045","task_key":"default--validation--47603","task_revision_id":"1","upstream_id":"47603","short_description":"HellaSwag validation 47603","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean your longchamp le pliage bag [title] Don't wash it in the washing machine. [step] The nylon material of the le pliage tends to \\\" bubble \\\" when handled improperly. Do not put them in the washing machine! They need to be hand-washed, gently.\",\"endings\":\"['[title] Lay the bag on the clean surface. [step] Wet both sides and the inside of the bag with water.', \\\"Dry off the bag thoroughly with a soft dry cloth. [title] If you do notice that the nylon material is touching the strap when holding it in place or you're worried that it might fall off, take it out of the washing machine.\\\", \\\"Always hand wash one bag at a time with the exception of the synthetic inner bag, which is waterless. Wash your bag in the washing machine, not on it's surface.\\\", '[substeps] Setting the temperature to \\\" soft cycle \\\" is more likely to be effective than washing the water or cleaning the bag as usual. [title] Make sure you wash it at the proper temperature.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}