{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a815977-6a5b-55d7-81e5-b0bf851c4419","task_key":"default--validation--14207","task_revision_id":"1","upstream_id":"14207","short_description":"HellaSwag validation 14207","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make canape of filo wrapped asparagus [title] Pre-heat your oven to 220 ° c (428 ° f) or gas mark 7. [title] Blanch the asparagus. [step] Sprinkle salt into a saucepan of boiling water.\",\"endings\":\"['Stir the asparagus filo into the boiling water. [substeps] Blanching asparagus ensures that they are crunchy.', 'Add lentils and onion and cook until they melt. [substeps] Blanching is also a good way to cook the asparagus.', '[substeps] Put the bowl containing the asparagus into the boiling water in the preheated oven. While it boils, simmer the asparagus.', 'Add the asparagus spears and boil for about 30 seconds (or until the water boils again). Use the slotted spoon to remove the spears from the water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}