{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0df97af-032b-5b67-81fe-a35491aa2606","task_key":"default--validation--43551","task_revision_id":"1","upstream_id":"43551","short_description":"HellaSwag validation 43551","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to repair a scrape on faux leather shoes [title] Wipe down the area with a soft cloth and warm water. [step] Then dab the area with a little distilled white vinegar. Treat a small part of the affected area with the vinegar.\",\"endings\":\"['[title] Add a bit of water, salt, baking soda and a bit of dish soap to the mixture and allow to sit for 1 to 3 minutes, then apply a bit more dish soap. [step] This will prevent the stain from washing off.', 'Repeat the process with fresh lemon juice if it helps reduce the appearance. You can skip this step if you do not mind paying extra attention to the stained area of the leather.', '[substeps] Take a small dab of distilled white vinegar on a paper towel and apply it to an area around the scratch. The vinegar may cause the area to slightly swell.', 'Repeat this several times, then wrap it in a cloth or a towel to keep it warm. [substeps] The vinegar will soften the leather during this step.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}