{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e8c10292-1336-5e17-8e68-f580dc1d2542","task_key":"default--validation--39201","task_revision_id":"1","upstream_id":"39201","short_description":"HellaSwag validation 39201","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wash jeans by hand [title] Fill your bathtub or utility sink with cool to lukewarm water. [step] Make sure the area is clean and that you run enough water to fully submerge your jeans. Wash dark jeans and brand new jeans in cooler water to prevent fading.\",\"endings\":\"['[substeps] Another option is placing the jeans in the dye water bowl that came with the dye, saturating them enough that they are safe to proceed. Follow all recommended temperatures for washing dark jeans, starting from the heaviest setting.', '[substeps] Cold water might be necessary for damp jeans. [title] Soak your jeans in the washing machine for a short amount of time.', '[substeps] Skip this step if you plan to wash the jeans separately. As a rule of thumb, use cool water to help with washing them separately.', '[title] Add a small amount of mild, liquid detergent to the water. [step] Swish the water a bit so the soap is mixed in well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}