{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7f9e501-db8a-5c46-acdc-e773e3fb2e82","task_key":"default--validation--20814","task_revision_id":"1","upstream_id":"20814","short_description":"HellaSwag validation 20814","config":"default","split":"validation","body":"{\"activity_label\":\"Preparing pasta\",\"ctx\":\"A bit of oil is then placed in a skillet and canadian bacon and onions are being fried. Next, more ingredients are added with pasta and egg yolks. finally\",\"endings\":\"[', the sauce is done and all of the ingredients are compiled and cheese is added to the dish as it is positioned on the plate.', ', turkey is scooped into a bowl and blended together with chocolate chips and turkey.', ', the chicken sandwich is powdered mix and served up and served in plastic bags.', ', various juices are squeezed and mixed into the pan then they are set on drying racks.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}