{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c21531b1-2897-564f-adcf-5404677b66e6","task_key":"default--validation--7318","task_revision_id":"1","upstream_id":"7318","short_description":"HellaSwag validation 7318","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean mold in shower [title] Make a mold-killing solution. [step] Pour ½ cup (120 ml) of vinegar into 1 quart (1 liter) of warm water. Use distilled white vinegar.\",\"endings\":\"['The solution will evaporate because the mold is gone. Stir with a spoon to mix in the vinegar.', 'Then mix in ¼ cup (59 ml) of borax. Mix the ingredients together in a pitcher until they are well-combined and the borax is completely dissolved.', 'Stir to combine, then pour it into the mold-killing solution. Using a small can opener, cut and mash the mold-killing solution.', 'After the vinegar forms, get down onto the floor and dunk your face into the mixture. Bring the solution up high until you see bubbles that bubble up to the surface.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}