{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad5f0397-8623-5c4b-a3c9-70df777d4ade","task_key":"default--validation--46569","task_revision_id":"1","upstream_id":"46569","short_description":"HellaSwag validation 46569","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to plant begonia bulbs [title] Purchase tubers (bulbs) by mid winter. [step] Since begonias need to be planted in early spring, the best time to find a good selection is during the winter. There are hundreds of varieties to choose from, but all tuberous begonias require the same care.\",\"endings\":\"['[title] Purchase dark, sugar-rich potting mix and medium-rich soil. [step] Tubers should grow in full shade and rot quickly.', 'You may want to look through a catalog or browse online to determine what type of begonias to plant. [substeps] There are over a thousand species of begonias, known to us, and not all of them are tuberous.', 'Lilies, begonias and poinsettia are grown in 6 varieties, a minimum of six if the plant is growing multiple bulbs. Most potting plants and houseplants will produce 2 bulbs each year.', \\\"[substeps] Winter plants may be applied after the northern hemisphere, such as other kinds of plants. Because the cold causes the roots to emerge, the flowers may need to be removed because that's the temperature the plant tends to be in.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}