{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5598582b-91f8-5524-aa43-cf393bad8aba","task_key":"default--validation--23386","task_revision_id":"1","upstream_id":"23386","short_description":"HellaSwag validation 23386","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to spot clean suede [title] Brush the dirt spots on your suede. [step] You can find brushes specifically made for cleaning suede at stores like target or walmart. However, a toothbrush or generic clean scrub brush will also work.\",\"endings\":\"[\\\"You don't want to use a very stiff brush as fine grit would contribute to scratches and grooves. [title] Rinse the brush under running water.\\\", '[substeps] The bristles on your brush should be fairly stiff to clean well. This method works well for getting the scuff and dirt marks off of suede shoes.', 'Start from the top edge of your suede and move down the suede slowly. Brushing the dirt spots on your suede is important because the dirt, dust, and debris may collect there.', 'Run the brush through bristles to get the dirt off. Be careful as the bristles are very sharp; stiff bristles can cut suede.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}