{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4fc3af71-ade0-5a02-94ec-18d5a2140b76","task_key":"default--validation--42488","task_revision_id":"1","upstream_id":"42488","short_description":"HellaSwag validation 42488","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to prepare for electrolysis [title] Get a clear cost estimate beforehand. [step] Know what you'll pay well before you show up for your appointment. The cost will depend on several factors, including the size of the area to be treated.\",\"endings\":\"['Many places charge by the minute and can give you a good estimate beforehand. You might even save money by scheduling multiple sessions over time.', 'Larger areas of the body will require deeper chemical signatures, so make sure all of the receipts you need for your procedure are printed. Better yet, ask the doctor to print it out for you so you can keep track of your monthly receipts.', \\\"For example, you may not have access to local water wells. Ask about how much you'll be working with in advance.\\\", 'Analyze the different costs and think carefully about what you actually want to pay before proceeding. [substeps] Add up any fees or costs before committing to the electrolysis procedure.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}