{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2b8cddd1-5b2d-5238-9dc0-83a14e78b685","task_key":"default--validation--35598","task_revision_id":"1","upstream_id":"35598","short_description":"HellaSwag validation 35598","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to care for a red eyed tree frog [title] Gut-load your crickets. [step] Before you feed your frog, you should feed its prey. Crickets are the primary food source for captive red-eyed tree frogs.\",\"endings\":\"['Help your local ecosystem by feeding crickets less active than other species. Make sure that you are feeding crickets that are passive and in teams rather than trying to feed them a lot of active so they grow smarter.', 'One or two days before you feed your frog, feed the crickets a mix of dried and fresh foods to ensure that they are full of essential nutrients. Some good foods for crickets include crushed dog or cat food pellets, commercially produced cricket feed, mixed seeds, alfalfa, lettuce, broccoli, carrots, and fruit.', 'If you have a frog that is able to eat, feed it 2-4 insects a day. Crickets feed on live crickets that are only available in the wild.', \\\"Feeding too many insects can damage the base of the frog's brain, so it's best to feed your frog non-insects, such as crickets, moths, grasshoppers, wasps, and insects with hair. Choosing insects and then eating them will both prevent the development of red eyed tree frogs.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}