{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9079f98a-3586-5c94-8b6b-a7a3a39a49ba","task_key":"default--validation--41097","task_revision_id":"1","upstream_id":"41097","short_description":"HellaSwag validation 41097","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to shop for makeup [title] Take stock of what you own and use. [step] Buying makeup is a waste of time if you're not going to use it. If you're the kind of person who only wears a little mascara, buying a large quantity of skin products isn't going to help you.\",\"endings\":\"[\\\"Likewise, if you already own four bright red lipsticks, one more isn't going to make a difference. [substeps] Before heading to the store, make a list of products you need or would actually use.\\\", \\\"If you bring you makeup without having any idea what's in it, you'll waste time. [substeps] Buy your product in one of three categories-lite, liquid and gel.\\\", \\\"Here are a few things you'll need : [substeps] Wide-mouth bags large batches of makeup will save you spending more time in the bathroom cleanser and moisturizer. Lip gloss and lanolin can add an extra layer of color to your skin.\\\", 'Try ordering cosmetics with several different colors, for example. If you just need foundation, consider getting one in a shade similar to what you usual spend money on.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}