{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f3d97812-92c4-5db0-b841-109e186d29c4","task_key":"default--validation--29410","task_revision_id":"1","upstream_id":"29410","short_description":"HellaSwag validation 29410","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to decorate a flower girl's basket [title] Choose a suitable basket. [step] Look in any craft store like joann fabrics or michaels for suitable baskets. Ideally, you want a basket that is dainty and small with a sturdy handle.\",\"endings\":\"['Your basket needs a spout and a flat, narrow outer plate that you can cover with a nice cloth. The basket needs to hold your flowers, jewel, and other baubles well, for it to stay upright.', \\\"If possible, choose baskets with small baskets or clasps that will fit into your flower girl basket. The basket should also be small enough to accommodate at least one of your flower girl's items.\\\", 'It should be large enough to contain the desired amount of flower petals, but small enough for a child to carry in one hand. [substeps] You should also make sure that it is a color that fits the wedding theme; if not, make sure that it is a neutral color or white.', 'Dig around in the basket and look for little pieces that fit your face. [title] Use a sheet of paper with a large colorful topknot facing the flower.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}