{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa04d9d9-02dd-5ce8-8452-ae28b3af06c5","task_key":"default--validation--25232","task_revision_id":"1","upstream_id":"25232","short_description":"HellaSwag validation 25232","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to prune patio tomatoes [title] Start at the bottom of the plant. [step] It can be helpful to start pruning your tomato plant from the bottom and working your way up. This will allow you to keep track of your progress and you'll be able to clearly see what needs to be removed as you work.\",\"endings\":\"['Carefully work your way up the plant, pruning any non-essential growth away from it. [substeps] Most of the pruning will take place at the bottom of the plant.', '[substeps] Prune to to remove hard to reach places. The hardest spots on your tomatoes are (but not always) the ones with the most dirt.', \\\"[substeps] It's a good idea to have someone else at the base of the plant so you know exactly where you're going. Work from the top up so that you can see exactly where you need to begin, as you'll be repairing damaged vines.\\\", \\\"[substeps] Cut back or remove any dead or diseased stems on your tomatoes so that you have a complete view of what's there. Dig out any roots or broken branches that didn't immediately fall off in the previous step.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}