{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f7914a5-06f8-595e-bc18-dd1c2737eb14","task_key":"default--validation--23925","task_revision_id":"1","upstream_id":"23925","short_description":"HellaSwag validation 23925","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to breed budgies [title] Decide whether you really want to breed budgies. [step] Breeding budgies takes time and isn't easy. You need to cash out on supplies and feed, check on the budgies often, clean the cage daily and set aside extra time for caring for the chicks.\",\"endings\":\"[\\\"You'll need to find a reputable breeder to market the farm and breed budgies for you. [substeps] Since budgies are never born of a mother, the only people that can breed yours are the chicks themselves.\\\", '[title] Find a suitable room to breed budgies. [step] The room should be quiet, protected from predators including pets, have no disturbances and the only person entering the room should be you.', \\\"Fertile chicks are ideal breeders, since they can mature when they are not nursing. It's also a good idea to hire babysitters.\\\", \\\"[substeps] Budgies develop quickly and i don't want to wait until they grow bigger to breed them as it's easier for embryos to grow. [title] Set up habitat for your budgies.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}