{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4e96c13f-2cbf-569b-9b51-d93ba1137cbf","task_key":"default--validation--47962","task_revision_id":"1","upstream_id":"47962","short_description":"HellaSwag validation 47962","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to start a vineyard [title] Learn about the process of growing grapes. [step] The first step is to do your research, and know exactly what you're getting into. For the first several years of vine growth, you may see minimal or no fruit production, and during this time, the vines are vulnerable to a number pests, fungi and diseases.\",\"endings\":\"['Expect to produce several harvests in this time. [substeps] Make sure that you are aware of any possible pests.', 'As a result, you may want to learn about some of these diseases and get more than an idea of how to grow each species. [title] Rake the pests out after they die out.', 'Growing healthy grapes is a huge time investment. [substeps] Visit or contact a local vineyard, and ask to spend time working the vineyard with them.', 'Harvesting grapes is the second crucial part of growing grapes. [title] Purchase grapes from a local wine store or online.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}