{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4945be85-4aa8-56e8-a499-707bba36bfb6","task_key":"default--validation--44428","task_revision_id":"1","upstream_id":"44428","short_description":"HellaSwag validation 44428","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to make rasgullas [title] Heat the milk and bring to boil. [title] Cool the milk for a couple of hours. [step] Remove the cream layer.\",\"endings\":\"['Drain the milk and return to a pan the size of a hockey ball. Turn the heat on low and bring the milk to a boil.', '[title] Reheat the milk and bring to a boil. [title] Add the citric acid dissolved in some water.', \\\"Turn off the stove and allow the mixture to cool to room temperature. It's now ready to be used for the rasgullas.\\\", 'Whisk in the cream of your choice and heat a little on both sides until it thickens. [title] Combine the sugar and water in a small saucepan.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}