{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"71ce6aa6-ed1b-5d55-bf78-cb8149f56514","task_key":"default--validation--21974","task_revision_id":"1","upstream_id":"21974","short_description":"HellaSwag validation 21974","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to freshen smelly shoes [title] Look for the cause. [step] Stop and examine your shoes before you try to treat the odor. If you have damp or damaged insoles, either take them out and dry them, or purchase insoles that are specially formulated to prohibit bacterial growth.\",\"endings\":\"['[title] Dry out your shoes by putting them near a heater or in a sunny spot. [step] Take out the laces and lift the tongue up and out to dry the shoes out faster.', 'If you had them professionally cleaned, you should be able to kill the bacteria that causes stains. [substeps] If you need to clean them more frequently than usual, buy quality insoles.', '[substeps] Check the centers for disease control and prevention (cdc). [title] Check to see if your shoes have been thoroughly cleaned.', '[title] Wear, though not a slip-on shoe, breathable shoe. [step] If you have feet that sweat a lot, a breathable shoe is your safest bet.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}