{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02e740ba-fef3-5902-9598-9e5d89c78a41","task_key":"default--validation--26419","task_revision_id":"1","upstream_id":"26419","short_description":"HellaSwag validation 26419","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to kill creepers in minecraft pe [title] Get the best sword you can find. [step] If you don't have any you might want to make one before this fight. [title] Move away from your home or anything you have built in your world.\",\"endings\":\"['[step] You want this to be a time that you cannot be bothered playing, so your better off if you have to kill creepers. [title] Find the latest white sword you have going in minecraft pe.', \\\"[step] If this battle ever ends, if you get caught (who knows...maybe monsters aren't caught here), enemies will die if you leave your house behind. [title] Do not advance unless you are warned by a monster.\\\", \\\"[title] Use fake bats to karp at these locations. [title] If you are going to use real bats you'll need to boss kill them in order for them to finish up.\\\", '[step] If a creeper blows up near it, it could end up destroying a lot of your items and materials. [title] Find a creeper and lead it to some flatter ground.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}