{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6b9645a8-7161-5560-ae27-f6e9a0bf7d4d","task_key":"default--validation--826","task_revision_id":"1","upstream_id":"826","short_description":"HellaSwag validation 826","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to add a simple touch of lolita to your everyday outfit [title] Wear a frilly blouse. [step] Most lolita styles include a blouse with puffy sleeves and ruffles. The sleeves can be long or short, but for a traditional approach, they should cover the shoulder.\",\"endings\":\"['The sleeves should be long enough to extend off the shoulder. If the blouse does not have puffy sleeves, skip this step.', 'For an elegant, ornamental look, wrap the blouse around your arm. Iron the sleeves and add embellishments on the buttons if desired.', 'The most traditional blouses also have a button front and rounded collars. [title] Try on a bell-shaped skirt.', 'Look for blouses with lace along the neckline or straps in the front. Try a blouse with ruffles above the bust, trim trim around the waist (blouse), and buttons in the front.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}