{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2935b99f-2b99-548b-9627-d1576333236e","task_key":"default--validation--35679","task_revision_id":"1","upstream_id":"35679","short_description":"HellaSwag validation 35679","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to pick dragon fruit [title] Expect the fruit to ripen about a month after it flowers. [step] Generally, dragon fruit will ripen about a month after it flowers. Check the dragon fruit daily for any flowers and use the flowers as a gauge for ripeness.\",\"endings\":\"['[substeps] Mid-spring on a weekday should be the best time to pick dragon fruit, as yellow blossoms develop outward over the course of several weeks. Fall flowers can be picked in late spring or early summer, but flowering flowers usually will bloom best in late august.', '[title] Check for a saturated fuchsia color to determine ripeness. [step] Watch for the color of the fruit to change from green to fuchsia or yellow, depending on the variety.', \\\"You'll also need to wait a few days to see how many large flowers the dragon fruit has. [title] Harvest dragon fruit in mid-spring.\\\", '[substeps] Still, use the events you observe as an example when picking dragon fruit. The fruit will ripen around the fall holidays.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}