{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c66b5e78-b9b9-5eee-a5a9-51da9f1d64c4","task_key":"default--validation--4721","task_revision_id":"1","upstream_id":"4721","short_description":"HellaSwag validation 4721","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to train shihpoos [title] Start crate training early. [step] Since shihpoos can be stubborn, you should start training your shihpoo as soon as you bring him home. If you wait too long, his stubbornness could add a frustrating challenge to your training.\",\"endings\":\"[\\\"Start training then at mid-teens or early twenties. [substeps] Note, however, that you should not stay alone in a crate for long period of time, as it will cause too much stress on your dog's muscles and bones.\\\", '[substeps] The crate is not dog training and is not a place to socialize your shihpoo. The shihpoo does not like to be handled, so try making a routine out of your crate and expecting it to change some.', '[substeps] If you crate train your shihpoo, he may have the notion that he cannot get out of the crate. He may be very excited to get out and explore his new home.', '[substeps] Crate training will help keep your shihpoo out of trouble when you are not home. It will also increase the likelihood of him going to the bathroom outside, rather than in his crate or elsewhere in your home.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}