{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8dc9d57-bf27-5128-876f-b64a9de8386b","task_key":"default--validation--32910","task_revision_id":"1","upstream_id":"32910","short_description":"HellaSwag validation 32910","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to cut a limb from a tree [title] Wait until your tree is dormant in the winter to make your cuts. [step] Pruning your tree when it's dormant gives it time to heal before threatening pests arrive. If you have particularly cold winters, wait until the coldest part of winter is over before you prune your tree.\",\"endings\":\"[\\\"However, cutting your tree before the last light frost sets in can set the stage for unexpected pests. [substeps] While it may be tempting to prune your tree until it's freezing, try not to become obsessed with the idea of waiting until the time.\\\", \\\"[substeps] Don't cut too soon after temperatures drop below a three-month mark. Let it heal in two weeks or so.\\\", \\\"[title] Cut the branches back to their normal length. [step] Depending on your tree's type, when the tree is dormant it may be of no use to you.\\\", '[substeps] Pruning in the winter will also help your tree grow more during the spring. Avoid pruning your tree during the growing season or you could stunt its growth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}