{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc3e3c6e-71fa-5897-a309-72c1993c1ca7","task_key":"default--validation--33192","task_revision_id":"1","upstream_id":"33192","short_description":"HellaSwag validation 33192","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to create a bond with your hamster [title] Pick a cage type. [step] When looking for a hamster cage, you will come across a few different types of cages. Each type of cage will have its own benefits and drawbacks.\",\"endings\":\"['Review the following list to get an idea of which type of cage might work best for you and your new hamster : [substeps] Mesh or wire cages are easy to clean and have great ventilation. However, they may have small openings that can let smaller hamsters escape.', \\\"Each cage has a different personality, and so does a variety of physical features such as responsibility, endurance, and courage. [substeps] However, if you're choosing a cage specifically for a hamster, don't choose a cage that's too big for a hamster, as this will make it difficult for them to climb in.\\\", 'The most common types of cages are fur cages, wood cages, or wire cages. You will also want a large drop tray, as your hamster is likely to escape the cage if it is overcrowded.', \\\"They are very good at feeding and sleeping, they're good at playtime and games and they're good with puppies. [substeps] The cheapest type of cage is a compact cage with a specially designed paper mesh bottom.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}