{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9581522-3dcd-5146-81ca-7ebe6e999ee7","task_key":"default--validation--42182","task_revision_id":"1","upstream_id":"42182","short_description":"HellaSwag validation 42182","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to bake codfish [title] Preheat your oven to 400 degrees fahrenheit. [step] If using a metric oven, this is 205 degrees celsius. [title] Clean the fish under cold water.\",\"endings\":\"['[step] Use a tablespoon to score each side, looking for fins and bones. [title] Rinse the fish, then dry it completely with a clean paper towel.', '[step] You can use a sponge, a spoon, or a foil tray. This should remove both the skin and the flesh of the fish.', \\\"[step] To maintain its freshness, you need to remove all debris and filth from the fish. It's imperative that you keep clean, fresh fish from bacteria and other fibroil elements.\\\", '[step] Once the cod has been cleaned, pat it gently with a clean paper towel to absorb any excess water, especially if the cod has been frozen and was recently defrosted. [title] Tear off a piece of aluminum foil twice as long and twice as wide as each piece of cod.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}