{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36abaf47-42db-5dea-9bdd-5183de197aad","task_key":"default--validation--23457","task_revision_id":"1","upstream_id":"23457","short_description":"HellaSwag validation 23457","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make indian red sauce pasta [title] Make a tomato paste. [step] Make criss-cross cuts over the tomatoes and remove their wooden ridges. Boil the tomatoes until the skin peels off.\",\"endings\":\"[\\\"Once they're sufficiently raw to taste, drain the tomatoes. Dump the tomato paste into a bowl and then stir in 2 diced tomatoes.\\\", 'Drain the tomatoes by placing them in water. Throw away the skins and discard the flesh.', 'Then, drain, but do not remove, the core. Trim off the tomato skins and let them cool a little.', 'Drain the water and peel off the skins. Add these tomatoes into a blender or grinder, along with 1 small cinnamon stick and 2-3 cloves, and grind it into a fine paste.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}