{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3176a10e-ecc2-553f-83ec-9b96af3ff223","task_key":"default--validation--39915","task_revision_id":"1","upstream_id":"39915","short_description":"HellaSwag validation 39915","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make cheese baingan (aubergine) ki kachori [title] Sift 2 to 3 cups of wheat flour into a bowl. [step] Add the gram flour, all purpose flour, followed by 1 teaspoon of oil, the ajwain, a pinch of salt and a pinch of baking soda. Add water to the ingredients in the bowl and knead it to make a soft dough.\",\"endings\":\"['[title] Add 1 tbsp of salt for each tbsp of water used. [step] Keep kneading with your fingers till all 3 ingredients are thoroughly incorporated.', 'This should form a dough that fits into the bowl. [substeps] To do this, lightly flour the surface of the bowl with your hands and roll out the dough to form a ball.', 'Once done, keep it aside for 15 minutes. [title] Place the round slices of brinjal (eggplant/aubergine) in a bowl.', 'This dough is creamy and light enough to melt into butter. [title] Separate and shape the dough into a circle.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}