{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3ad54cf9-e760-596f-8d98-6677e3c12846","task_key":"default--validation--43407","task_revision_id":"1","upstream_id":"43407","short_description":"HellaSwag validation 43407","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a homemade healthy face wash [title] Combine all of the ingredients. [step] In a medium-sized bowl, mix together ½ cup (45 g) of rolled oats, ¼ cup (59 ml) fresh lemon juice, ¼ cup (59 ml) water, and ½ tablespoon (11 g) of honey. Use a spoon to stir the mixture until all of the ingredients are well combined.\",\"endings\":\"[\\\"[substeps] To avoid possibly adding chemicals to your face wash, it's best to use filtered water. You can use any type of honey that you like, but an organic variety is usually best.\\\", '[substeps] You can use most of the oats mix, but you may need a dash of fresh lemon juice. [title] Swish the mixture gently in your mouth.', '[substeps] You can substitute honey for the lemon juice if you prefer. Always ask your provider which type of honey you can use for your face wash.', '[substeps] The oats will give the face wash a dry, grainy appearance. Avoid using natural ingredients like rice, cashews, and raw fruits or vegetables that may irritate your skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}