{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fcd2d6a1-0ca4-5d94-9934-216f78707896","task_key":"default--validation--17886","task_revision_id":"1","upstream_id":"17886","short_description":"HellaSwag validation 17886","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look good on a limited budget [title] Look for deals in stores. [step] Most of the time stores are having sales to get rid of clothes that are not selling or just to make way for the new clothes. Shopping for deals is good.\",\"endings\":\"[\\\"Always be sure to check out your local thrift store, like goodwill or salvation army. If you're a humanitarian, pay it forward and donate your nice-looking, unused clothes.\\\", \\\"If you can't find designer clothing or online clothing ideas, shop around for clothes that seem like deals. Look in specialty clothing stores or through sites like any other retail store.\\\", 'It can be expensive, but looking good will make your competition less willing to step on your toes. [substeps] Make a shopping list of things on your list which you will buy most at a reasonable price.', 'Paying for a new outfit, then taking off the clothes, is especially good. [substeps] Some stores have coupons for new clothes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}