{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb382310-89e0-540e-ba79-be1e388cd6a9","task_key":"default--validation--47289","task_revision_id":"1","upstream_id":"47289","short_description":"HellaSwag validation 47289","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to bake kabocha squash [title] Cut the stem off the top of the squash. [step] Place the tip of a sharp kitchen knife 1 inch (2.5 cm) from the stem of the squash. Penetrate the skin of the squash and cut around the stem.\",\"endings\":\"['Lift the peel off the top of the squash by turning the knife counterclockwise and keeping pressure on the peel. Tear off the top of the squash by turning it sideways with a sharp knife.', \\\"Pull the tender sections from the stem and discard the central core. [substeps] Sometimes you'll need to water the squash to dry it out.\\\", 'Then, pull up on the stem to remove it. [substeps] You need a sharp knife to cut through the thick skin of the squash.', '[substeps] If possible, have someone else cut the squash up to ensure that it is safe to be cut into pieces. [title] Boil the squash to turn it into bite-sized pieces.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}