{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d083678-3405-5da5-8385-fd40eed3ed3f","task_key":"default--validation--44044","task_revision_id":"1","upstream_id":"44044","short_description":"HellaSwag validation 44044","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to tell if wine is corked [title] Smell the wine. [step] If wine is corked, it will have an odor that you wouldn't expect from a good wine. It may smell musty, or like dank towels, wet dog, wet cardboard or newspaper.\",\"endings\":\"['[substeps] Note that your first sniff is more reliable than later sniffs. Trust your first whiff! Wine becomes corked when it is exposed to tca or \\\" 2 , 4 , 6-trichloroanisole \\\", a naturally occurring compound that may be found in the cork of the wine bottle.', 'A firm cork should be clinking up every minute or so, as it develops. A soft, swish tasting wine should have a heavier, richer flavor than the bottle.', '[substeps] If the wine is corked on a cork with a strong odor, it will need to be corked! If the wine is corked on a cork with a flat middle of air, it will have a sour smell. [title] Taste, smell, and taste the vinegar.', \\\"Let your eyes examine the wine's flavors and aroma to check that it is fresh enough to be corked. If it smells appealing to you, you should have plenty of grape corks printed into it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}