{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ee0d1e6-907c-593c-ace3-d51d9ddff1c6","task_key":"default--validation--30623--variant-4688ba75ed56","task_revision_id":"1","upstream_id":"30623--variant-4688ba75ed56","short_description":"HellaSwag validation 30623","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to dress cool in high school (for guys ) [title] Decide what style you want to aim for. [step] Consider the styles below : [substeps] Casual: casual style basically consists of simple jeans, t-shirts and sweaters. This is the safest and most common style among high school boys.\",\"endings\":\"['Sporty: as the name suggests, this style is most favoured by members of school sports teams. The sporty style includes track pants, sneakers, sports team tops and varsity jackets.', \\\"Casual style also involves short skirts, shirts and pants that you can bring with you if you want to sound cool. Since these rarely get worn daily, high school guys don't care if they do.\\\", \\\"Loose style generally consists of skinny, low cut, long sleeve cardigans, wayfarers, and other items that complement other casual but comfortable wear. Make sure to avoid styles that aren't revealing or stereotypically emo/dressy, however.\\\", 'You probably will cut your jeans a little too short but still dress the same. Oxford casual: casual casual style consists of pants, a casual top and nice shoes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}