{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d3cf1ee4-6eee-5bf3-8082-22b50b4efdb8","task_key":"default--validation--25622","task_revision_id":"1","upstream_id":"25622","short_description":"HellaSwag validation 25622","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wash your rings [title] Gather the needed supplies. [step] You will need : [substeps] A toothbrush hot water toothpaste rubbing alcohol paper towel a small cup [title] Decide which ring you want to wash. [step] Turn the faucet on and have the running water be warm.\",\"endings\":\"['You may also want to use a hot rag or milk jug to soak your rings in the hot water. Excess water attracts dirt and grease, which can make piercing application less effective.', 'Place the ring in the water and start rinsing it. Make sure the solution is mostly water and not vinegar water, as vinegar also has formaldehyde in it.', 'Clean the ring between your fingers, not across the coin. If necessary, use a label or marker to record the length.', 'Warm water is good for killing bacteria and cleaning stains. Make sure it is not unbearably hot as it can burn your skin and damage your ring.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}