{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7726a65e-003a-5585-a6cc-19d19885f2b0","task_key":"default--validation--46099","task_revision_id":"1","upstream_id":"46099","short_description":"HellaSwag validation 46099","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to safely clean a diamond ring [title] Make a soap and water mixture. [step] Squirt a little dish soap into a bowl. Fill the bowl with warm water.\",\"endings\":\"['Use approximately one part soap and six parts water. [substeps] You can begin pouring water into the bowl at any time of day.', 'Swish it around a bit to produce some suds. [substeps] Use a gentle dish soap, preferably one made with natural ingredients, to avoid harming your ring with chemicals.', 'Put your ring in the water and let it soak for as long as you like , 5 minutes. This is the soap solution for cleaning a diamond.', 'Then, stir in a few drops of dish soap. [title] Dip your ring into the soapy solution.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}