{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d74722a5-9677-5d7d-a7d4-d3869e394fca","task_key":"default--validation--35998","task_revision_id":"1","upstream_id":"35998","short_description":"HellaSwag validation 35998","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean a ceramic sink [title] Dampen your sink. [step] Before you apply the baking soda, lightly dampen the basin of the ceramic sink. You can do this by allowing the sink to fill with water and then draining it, or using your hand to splash water around on the sides of the basin.\",\"endings\":\"['Make sure that no standing water is left in the sink, as this will dissolve the baking soda and render it useless. [substeps] Also dampen the sponge that you plan to clean the sink with.', \\\"[substeps] This step isn't necessary for all ceramic sinks. The baking soda and all the other ingredients will dissolve any remaining residue, leaving behind the bowl-like residue.\\\", 'Use a damp cloth to wet the inside of the basin while the sink is in the process of cleaning. [substeps] You can also use a cloth to help drip water into the basin, making cleaning easier.', '[title] Apply a layer of paste into your sink. [step] Make sure that you : [substeps] Blot in the paste with a clean, white cloth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}