{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5f3f3b0-3770-56be-9455-55fbc4a0523f","task_key":"default--validation--40977","task_revision_id":"1","upstream_id":"40977","short_description":"HellaSwag validation 40977","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to prune persimmons [title] Choose the right time. [step] A persimmon is a wonderful tree to have in your yard or orchard. They are visually pleasing and bear a delicious fruit.\",\"endings\":\"['[title] Choose a place for eating as raspberries and apples grow best. [title] Find perennials or berries for embellishment.', 'Take care to pick the right time to prune your persimmon. [substeps] Prune your tree in late winter or early spring.', 'However, a persimmon tree will grow poorly and may need to be pruned sooner rather than later. Ideally, wait until evening to start pracking the tree.', 'Pruning persimmons in the early morning or evening usually makes it easier for the persimmons to blossom and produce. [title] Read the pruning instructions closely.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}