{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e15e64c-48cf-55d9-aec0-05fb7d756e91","task_key":"default--validation--45048","task_revision_id":"1","upstream_id":"45048","short_description":"HellaSwag validation 45048","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to walk a guinea pig on a leash [title] Make sure the harness isn't too tight. [step] If you put on a harness, make sure it is loose enough. You should be able to put a finger or two between the harness and the guinea pig.\",\"endings\":\"[\\\"If it feels uncomfortable, put it off. That way the guinea pig won't be able to run away.\\\", 'If it is too loose, then the dog may buck, or lunge. [title] Practice walking with your pet on a leash.', \\\"[substeps] Try to move very slowly and avoid getting too close to your guinea pig. Don't yank or pull on the harness near your guinea pig's mouth.\\\", \\\"If it's too tight, it could harm the guinea pig's back over time and chaff under his armpits. [substeps] It may not be a good idea to put a harness on at all.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}