{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24939384-e2c9-5130-ae90-1106a10811b0","task_key":"default--validation--20078","task_revision_id":"1","upstream_id":"20078","short_description":"HellaSwag validation 20078","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to give a sensual foot massage [title] Prepare a clean, decluttered room. [step] You can give a sensual massage anywhere, really, but the room should be soothing. This means removing any extra junk from the room so it looks clean.\",\"endings\":\"['If you exercise at the very time you will be giving a sensual massage, go for it. [title] Pour warm water into a large pot.', \\\"Go for windows, mirrors, and anything else you won't mind picking up later. [title] If you're using carpet, you'll need to cut it to size.\\\", '[substeps] You also want to clean your entire body as well as the area where you are massaging. If you are massaging your feet because of a spill on your, make sure that the stair railing is clean and neat and remind your family that the mess is for their own good.', \\\"You should also consider laying down a clean, cotton sheet wherever your partner will be laying down for their massage. [substeps] The sheet will feel good on your partner's skin, and it will be easy to clean afterwards in case any body lotion or oil gets on it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}