{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2f2847ca-2693-576a-bfbb-ba56227a436d","task_key":"default--validation--28707","task_revision_id":"1","upstream_id":"28707","short_description":"HellaSwag validation 28707","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to make zoodle soup [title] Use a spiralizer. [step] This is the preferred tool for many people because it's the fastest and easiest to operate. Unfortunately, it's also the largest and most expensive.\",\"endings\":\"[\\\"The bowels that create zoodles can take a toll on you with long-term use. As a result, zoodles can become extremely hard to work with, especially if they're left in the grinder for too long.\\\", 'Once you begin to see results with the technique, you may want to start the process over and try another batch. [substeps] Spiralizers can be very expensive, so make sure to check reputable shops before making your own.', 'To use it, simply wash your zucchini and slice the ends off. Place the zucchini next to the blade and spin.', \\\"It's quite discreet, and makes the process much safer for you. [substeps] Spiralizers are small tubes used to filter out milk and water.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}