{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d6999ee-8e45-55f1-b867-1ef460cadd6c","task_key":"default--validation--17781","task_revision_id":"1","upstream_id":"17781","short_description":"HellaSwag validation 17781","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to organize gardening tools [title] Hang a bike basket on the wall to hold a variety of small items. [step] Bike baskets are designed to be easily hung, so they make an ideal option for hanging from the walls in your garden shed. Use them to store miscellaneous items like gloves, seed packets, pegs, or other small items that might be laying around otherwise.\",\"endings\":\"['[substeps] Avoid baskets with handles that might break and fall easily. If the basket does hit the wall, add sturdy tape or foil to the rim to better hold it in place.', \\\"It may seem a little heavy, but using them will help prevent your tools from tipping over when you have to use them so they don't fall out of reach. [substeps] Cycling helmets are an example of a bicycle basket.\\\", '[substeps] Handhang your outdoor shed baskets from the ceiling, wall, and ceiling so they can be easily hung from a tree or hanging tools. [title] Place the basket on a support structure, like a wall, to keep the baskets elevated on top of it.', '[title] Hang a rake head upside-down from the wall for a unique hand tool holder. [step] A steel rake head makes a unique holder for your garden gloves, trowels, spades, and other small objects.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}