{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a10794be-3e84-595d-a919-cb09e178049b","task_key":"default--validation--14595","task_revision_id":"1","upstream_id":"14595","short_description":"HellaSwag validation 14595","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to grow bird's nest fern as an indoor plant [title] Plant the bird's nest fern in good quality potting soil. [step] Be sure to use soil that permits excellent drainage as this fern does not like waterlogged feet. [substeps] Also ensure that the pot is able to drain water with ease.\",\"endings\":\"['Filling the saucer with marbles, glass pebbles or real pebbles can help make drainage better, as well as allowing for evaporation to create humidity. [title] Water very generously during warmer months and in warm indoor environments.', 'Add mulch to the planting site to deter rot. [title] Set aside at least 8 hours before heading out to shake branches or brush for fertilizers.', 'Do not fertilize it with cat litter, wood chips, garden compost, or manure as the species can rot or die during this time. [title] Allow the fern to recover for 24 hours.', 'You want the fern to retain water until it is ready to grow to avoid becoming too soggy. [title] Put on gardening gloves to keep ferns wet.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}