{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44483d95-866c-550f-a38f-5fcb58852d0e","task_key":"default--validation--29578","task_revision_id":"1","upstream_id":"29578","short_description":"HellaSwag validation 29578","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to grow yams [title] Start growing the sprouts. [step] Yams aren't grown from seeds like most other vegetables-they grow from slips, which are derived from the sprouts of adult yams. To grow sprouts, cut a yam in half and submerge one portion in a glass of cool water.\",\"endings\":\"['Right before the plants start sprouting, dig out a piece of the stem and then tuck that beneath the water to make it flush with the bottom of the glass. Repeat this process until you have about two inches (5 cm) of stem.', 'Halfway through the soaking process, seal the other half in a commercial water bucket. [substeps] Think about using cans, tupperware buckets, and other containers that are big enough to hold several yams at once.', \\\"You should see the seeds sprout. [substeps] If you can't, cut small sections back and keep the sprouts in the glass of cool water.\\\", 'Insert toothpicks at three points around the middle of the yam and suspend it over the container, half-submerged in the water. [substeps] Make sure the yam you choose looks healthy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}