{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0ab5fe44-e8a4-596c-9b46-538178256974","task_key":"default--validation--14558","task_revision_id":"1","upstream_id":"14558","short_description":"HellaSwag validation 14558","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to restore thick dried out nail polish [title] Get acetone from a beauty supply shop. [step] Most beauty shops sell acetone. You may also be able to find it in a drug store that sells beauty supplies.\",\"endings\":\"['[title] Cut the end off of your normal nail with a nail file. [step] Not only do these edges make the polish tacky, but they leave behind a very nasty odor.', 'Do not dilute it in water, as it is sodium bicarbonate (acetone) which can be harmful if spilled. Make sure you check with your specialist if you are unsure.', 'If you do not have acetone on hand, you can easily apply acetone to your nails yourself. [title] Hold your hand out to the side.', 'Once you have your acetone, add a few drops to a container. [substeps] Acetone is acidic, so you will need a container it will not eat through.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}