{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9868f25-782f-5194-8616-85cd12702b8c","task_key":"default--validation--22544","task_revision_id":"1","upstream_id":"22544","short_description":"HellaSwag validation 22544","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to be vegetarian on a budget [title] Select protein rich affordable foods such as : [substeps] Dried foods such as: beans, rice, quinoa, lentils, couscous, polenta or dried peas. These can be purchased in bulk at very good prices and will store for a long time provided, if you store them properly. They go a long way and are a good budget food.\",\"endings\":\"['[title] Make vegetarian protein from pellet protein. [step] Dry protein is the protein that you put into your meat daily, you can wrap and freeze it in chickpeas.', 'They are so costly though that you should like them just fine. Try to stick to whole foods at all times.', 'Protein enriched foods such as: beans, peas, nuts, dark meat poultry, fish such as salmon, trout, haddock, herring or tuna. They should not be more than 3% protein.', 'Tinned beans and ingredients such as chickpeas, tinned chopped tomatoes etc can be very economically purchased. See stock a vegetarian pantry on wikihow for more information on different products.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}