{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23bbf8df-9c7a-5e27-926c-c90cb965545c","task_key":"default--validation--39511","task_revision_id":"1","upstream_id":"39511","short_description":"HellaSwag validation 39511","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to kill black ants [title] Buy commercial ant bait. [step] Using a commercial bait is a very effective way to kill black ants. Ants are attracted to the bait, which is a mixture of poison and a sweet substance.\",\"endings\":\"[\\\"If you find the bait stuck in the ants' hole, you can try to move the bait closer to the hole. Mosquitoes are repelled by the bait, so they'll leave you alone.\\\", 'The ants eat some bait and bring some back to the nest, where the rest of the colony is also exposed to the poison. Over the course of one or two life cycles, the ants will be destroyed.', 'They will flock to the bait and sting, so when the bait is used, the ants will swarm and destroy the bait. [substeps] Unlike commercial bait, wasp bait is designed for you to use.', 'At a minimum, there are two types of bait that ants naturally use to hunt their prey. The one type is cyanide-which is a pesticide that attracts black ants.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}