{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f2ab6eed-94ac-5e91-9d53-5aa8d98b2826","task_key":"default--validation--41828","task_revision_id":"1","upstream_id":"41828","short_description":"HellaSwag validation 41828","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a cute clutch bag [title] Choose a rectangular placemat. [step] This method works best with fabric placemats. Choose something that looks interesting, preferably with embroidery.\",\"endings\":\"['[title] Slide the scarf end through the pendant hole on the placemat. [step] Squeeze the twist link and seal the ends shut.', 'Dyeing the placemat using only fabric placemats will make it resemble you is making the bag. [title] Add a single color of fabric placemat to the bag.', 'Something with a lining would work even better, and will give your clutch a more professional touch. Make sure that the material is sturdy and not flimsy.', '[title] Get a brown or light colored ribbon and wrap around the placemat for a cute look. [step] Add a bow to the ribbon for a fancy touch.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}