{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f354da74-2fd0-5558-a1ab-a47e3c27bae5","task_key":"default--validation--34136","task_revision_id":"1","upstream_id":"34136","short_description":"HellaSwag validation 34136","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to take care of diamond jewelry [title] Try not to touch the diamond. [step] If you love how your diamond sparkles, try to keep your fingers off it. As you touch it, you pass oil from your fingers to the diamond, which can make it look dull.\",\"endings\":\"[\\\"Ideally, you should also never touch the part of the diamond you're caressing. Be cautious when you run your fingers over the part of the diamond that's under weight, such as the side your bed is on.\\\", '[title] Avoid wearing jewelry that is too small or too large. [step] The wider and bigger the diamond, the larger its value.', \\\"[title] Remove your ring when working with chemicals. [step] Chemicals may make your bathtub sparkling clean, but they aren't doing your diamond ring any favors.\\\", \\\"This, in turn, causes sweat to build up. Consider keeping your hands away from the diamond when you're out in the sun and in humid conditions.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}