{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"084ea314-40f2-5c92-89e5-7b104d4ff83b","task_key":"default--validation--16698","task_revision_id":"1","upstream_id":"16698","short_description":"HellaSwag validation 16698","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to grow a garden in a bottle [title] Select your bottle. [step] The bottle should be large enough to allow room for plants to grow. Clean it and allow the bottle to dry thoroughly before using it.\",\"endings\":\"['[title] Cut your bottle so it is open on 1 side. [step] Leave most of the tube side open; one side is going to be extending over the bottle edge; use a pair of scissors to snip the tube apart.', \\\"The larger the opening, the easier it'll be to maintain the garden. [title] Turn the bottle right-side up.\\\", 'It should be sterile before using in a garden with young kids. [title] Use a bottle in a large pot.', 'This will give your plants ample time to grow. [substeps] If your bottle has holes in the top, cover those with some garden grit, crushed baking soda, or urine.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}