{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4aa0847-4461-50e0-8895-36f290e63920","task_key":"default--validation--11307","task_revision_id":"1","upstream_id":"11307","short_description":"HellaSwag validation 11307","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make perfume with essential oils [title] Learn the order of oils. [step] When making perfume from essential oils you'll need to follow an order: you start with a base oil, then have the middle tones, then finally the top notes. The top note is what you smell when you first smell your perfume, and then you are gradually able to smell your others.\",\"endings\":\"[\\\"You'll need to add oils in this order. [substeps] The top notes reach our senses first, but dissipate quickly.\\\", '[substeps] The base oils seem distant from the middle tone. Note that you should fill this space and leave this space bare.', '[substeps] Foundation perfume: start with a foundation to create your base fragrance. Then you can add the rest of your ingredients and do your best to blend them together.', 'In this case, go for the middle tones. [substeps] To produce the same strong fragrance for your favorite kind of perfume, fill a jar with the base oil.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}