{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17c55ebb-b7b2-51c5-b00a-a72adedfc630","task_key":"default--validation--21441","task_revision_id":"1","upstream_id":"21441","short_description":"HellaSwag validation 21441","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to buy tilapia fish [title] Research your local fish suppliers to determine where the best quality product is available from. [step] Ask the fishmonger at your grocery store or seafood supplier where their tilapia comes from and how it was grown. Because tilapia meat retains the flavor of the water the fish swam in, these facts are important to know.\",\"endings\":\"[\\\"[title] Visit a local store that sells fish you're not familiar with. [step] Do some comparison shopping and see how many fish you can buy at a given price.\\\", '[title] Keep the tilapia mobile to stabilize the water level. [step] Tilapia have a lightweight, aerodynamic form that allows them to travel in smooth and consistent waves.', '[title] Develop a budget for tilapia meat. [step] Tilapia meat is not calorie-dense and healthy, and should be compensated with additional calorie and protein sources when possible.', \\\"If your supplier doesn't know, this could be an indicator of poor quality and you should try another supplier. [title] Purchase tilapia live if at all possible.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}