{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a3287516-3cab-5c04-96c3-9719d98aad7a","task_key":"default--validation--13842","task_revision_id":"1","upstream_id":"13842","short_description":"HellaSwag validation 13842","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to plant pitcher plants [title] Use a premade mix specially created for carnivorous plants. [step] Using these mixes is highly recommended. Pitcher plants typically grow in damp soils that are devoid of most regular nutrients.\",\"endings\":\"['They evolved to catch bugs to fulfill their nutritional needs. Therefore, even the tiniest bit of chemical fertilizer is a recipe for disaster and most of the time will cause fatalities.', 'Consult your local garden nursery or cpa to determine if a specific mix will create a good environment for your pitcher plant. [substeps] Nearly any brand of bark can be grown in a dry, temperature controlled environment.', \\\"This, in turn, contributes to the breakdown of leaves and are the core of the garden's drainage. [title] Wash the plant in lukewarm water before transplanting.\\\", 'However, other types of plants may grow in colder soils. These kinds of plants do not require aeration but they can be grown indoors.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}