{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b2ddd4b5-c5f8-581c-830c-5f16db209944","task_key":"default--validation--43969","task_revision_id":"1","upstream_id":"43969","short_description":"HellaSwag validation 43969","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to deal with temptation [title] Recognize potential temptation. [step] Temptation is all about self-control, and tempting situations are often about the conflict between immediate gratification and long-term goals. For example, if you're on a diet, you may be tempted to eat a delicious piece of cheesecake that's beckoning you from the deli case.\",\"endings\":\"['Or if a friend offers to babysit your kids once school kicks off, you may be tempted to beat back or procrastinate. Ignore the craving when it strikes, and continue to eat for yourself.', \\\"As you meet in a bar at a club, you may be tempted to drink a sensible and healthy beverage. Understand what you're willing to enjoy in order to overcome temptations.\\\", 'Immediate gratification would result from giving in to temptation and eating the cheesecake. However, giving in would also interfere with your long-term goal of becoming more healthy by avoiding high-sugar foods.', \\\"But tempting diets encourage choices to be made that make food irresistible. Resist these temptations by first identifying potential temptations that won't give you what you want.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}