{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1aa2276b-45d9-5160-9af8-9adba7b482ad","task_key":"default--validation--18300","task_revision_id":"1","upstream_id":"18300","short_description":"HellaSwag validation 18300","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to age wine without a cellar [title] Learn the proper conditions for aging wine. [step] While it is aging, wine needs to be kept at a constant, chilly temperature and a relatively high humidity. It needs to be kept as dark as possible and disturbed as little as possible.\",\"endings\":\"['[substeps] The aging temperature is perhaps the most important criterion. Fifty-five degrees fahrenheit (13 celsius) is the ideal temperature.', 'Temperatures above 155ºf (39 ºc) are the most common, and should be kept at somewhere between 55 and 47ºf (0ºc to 10ºc). When the weather is dry and cold it is better to let wine rest in the refrigerator, or keep wine at room temperature.', \\\"Many people require a cellar while they'd like to age the wine, but most home wine makers are quite happy with this fix. [substeps] Windows: best protected in cold weather when temperatures are between 70 and 82 degrees fahrenheit (22 and 29 degrees celsius), and near closed windows where the temperature is a bit higher.\\\", 'Keep wine away from direct sunlight for roughly half of the winter years if you need to grow large grapes. In more severe cases, your winemaking business may have to wait longer in months than in years to replace the sunlight.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}