{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d72369c3-9e55-51a8-80f3-203401971127","task_key":"default--validation--48937","task_revision_id":"1","upstream_id":"48937","short_description":"HellaSwag validation 48937","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to keep a single ferret happy [title] Set aside play time every day. [step] Ferrets need a lot of attention, and they should receive a considerable amount of out of cage time every day. Make sure you have at least an hour to play one-on-one with your ferret on a daily basis, as too much cage time can lead to stress and illness.\",\"endings\":\"['[substeps] Make sure every night is play time for your ferret. Provide enough play time for both you and your ferret each day, as you may want to let each of them individually play for hours.', '[substeps] Ferrets are very intelligent and active, so you need to be engaged in play. This is especially important for single ferrets, who can get bored without a companion.', \\\"[title] Trim the ferret's nails. [step] Ferrets rarely gnaw at nails, nor do ferrets chew on very long and thin nails.\\\", '[title] Remove your ferret from the cage after meals and before bedtime. [step] This will reduce chances of your ferret crashing to the floor in his cage when he relaxes in his temporary den.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}