{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0fa5e542-ab8a-54ad-aa78-0743f264ee66","task_key":"default--validation--50039","task_revision_id":"1","upstream_id":"50039","short_description":"HellaSwag validation 50039","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to find inexpensive, good quality makeup [title] Browse through drugstores. [step] Drugstores are going to have the largest selection of inexpensive cosmetics. It is possible to find high quality products nearly equal to department store brands at a lower cost.\",\"endings\":\"[\\\"In fact, some of the drugstore brands are owned by the same companies that are sold in high end department stores. This doesn't always mean it is the same quality, but there are many hidden gems if you take a good look.\\\", 'The selection is not limited to certain types of products including : [substeps] Makeup-for contouring, protecting your face, or even a very subtle shade. Newer beauty stores like beauty stores have whole new stores to imax cheaper products.', 'Search at least five stores in the section. Once you see stores for sale, ask your friends and family if they have any disposable contacts, contacts, and a good quality makeup/combination.', 'Look for a beauty store, supermarket, or drug store, since drugstores usually only buy the latest brands. Nothing is higher quality, refined and disposable than the most expensive and cheap.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}