{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76e9092e-2cd2-52ab-9a23-b6adab23836d","task_key":"default--validation--7915","task_revision_id":"1","upstream_id":"7915","short_description":"HellaSwag validation 7915","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to create a twig wreath [title] Obtain a heavy wire clothes hanger. [title] Clip the \\\" hook \\\" off of the hanger with wire cutters. [title] Obtain a largish supply of smallish twigs.\",\"endings\":\"[\\\"[step] Try to keep the twig equal in diameter and length. Some trees work better than others and you should check for permission before trimming any foliage from property you don't own.\\\", '[step] The twigs will be the basis of the twig wreath. [title] Before choosing your twigs, get your twig.', '[step] Two twigs which you will use to wrap around the completed wreath. [title] Cut each twig approximately 1/4th of the way up.', '[step] Only once are you ready to plant. [title] Cut branches about 2 inches (5.1 cm) in length.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}