{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ecdf23e9-0ad8-5cac-8295-0748ad37281c","task_key":"default--validation--30858--variant-4d1b98640195","task_revision_id":"1","upstream_id":"30858--variant-4d1b98640195","short_description":"HellaSwag validation 30858","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to grow grape vines [title] Choose a type of grape. [step] As with any plant, certain types of grapes grow better in different areas and offer up different flavors and appearances. There are three general types of grapes: american, european, and muscadine grapes.\",\"endings\":\"[\\\"The' bat' variety is quite dark, with a hectic shape and a bit of texture. Popular grapes grow quickly for the same years and are often planted with white grapes and a rose color.\\\", '[substeps] Traditional grape grapes are made of rich grapes and have light yellow and orange coloring. If the grapes are not from the wild and you are looking for a more bitter flavor, look for grapes grown in known regions that are not based on the wild grape.', '[substeps] American grapes are naturally sweet and plump, with skins that are slightly crisped. European grapes grow best in warm, dry conditions with ripened fruits on a stalk.', 'American grapes grow best in warm, sunny climates like that of central california. European grapes are common in europe and northern parts of the us, and muscadine grapes are commonly found in the southern us.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}