{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df04bb9d-a895-5b5a-a6c6-5b72183415fa","task_key":"default--validation--30504","task_revision_id":"1","upstream_id":"30504","short_description":"HellaSwag validation 30504","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to personalize slippers [title] Decide on the color and position. [step] If you want to add your name to a slipper, this can be a nice personalized touch. However, before adding your name, stop and consider where you want to add your name and the color thread you will use.\",\"endings\":\"['Before you add your name to slippers, you need to make sure that the color and position you place your name are consistent. [substeps] If you go for clear or patterned designs, you may want to remove a layer of overspray or fabric before adding the name.', '[substeps] Choose a color that you like and goes well with the current color of your slippers. If you have yellow slippers, for example, they may look great with blue lettering.', 'You may also want to have a wallet or other form of identification to provide a nice touch. [substeps] Consider writing on your old socks or painting your feet or heels with a marker of your choice.', \\\"[title] Fold the paper so the front of the paper is facing you. [step] Place your bottom hand on the top and pull the paper down so it's right at your midline.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}