{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d852215c-015e-552b-a637-11fa4fbc401d","task_key":"default--validation--28720","task_revision_id":"1","upstream_id":"28720","short_description":"HellaSwag validation 28720","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress for a gala [title] Embrace the glamour at a white tie affair. [step] For a white tie event, look for a full-length ball gown in a seasonal color, or opt for an always-fashionable black gown. Go to a specialty dress store or search online, and have the gown tailored to fit you.\",\"endings\":\"['[title] Go with a traditional black tuxedo. [step] Black tuxedos are best for men.', \\\"Or select a fitted, strapless gown in a muted color to look more at-and-center for an event. [substeps] If you do go for a metallic color with your gown, you don't need to spend a fortune on a thousand-dollar dress.\\\", \\\"If you're on a budget, consider renting a gown for the event, which can be much cheaper. [substeps] Long gloves are common at white tie affairs, and can complement or match your dress color.\\\", \\\"Opt for a less expensive gown if you're the one who really wants to present the gala. Wear with some appropriate jewelry and wear a nice pair of heels.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}