{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5a0e607e-cf09-590b-8dd9-43bfd3f057ef","task_key":"default--validation--11327","task_revision_id":"1","upstream_id":"11327","short_description":"HellaSwag validation 11327","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to remove cactus needles [title] Take out standard cactus needles using tweezers. [step] If you can see a cactus needle clearly, grip the end of it with a pair of tweezers and pull it out. To avoid snapping the needle, do your best to remove it in a single, straight motion.\",\"endings\":\"['[substeps] A typical standard cactus needle will only have a single needle, so if you have someone else hold the needle by its other end, i would substitute a 2 \\\" (5.1 cm) colander. [title] Soak the needles for a day after removal.', \\\"Work in opposite directions from the spot where the needles were so you make more accurate cuts. [substeps] Don't use the tweezers to find the tip of the needle.\\\", 'Try to avoid biting the needle while it is still in the cactus. Use the tweezers to make this process much easier.', '[substeps] If the needle gets buried deep under your skin, locate the tip furthest away from the entry wound and gently push it out. [title] Brush off glochids using a nylon stocking.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}