{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d1be5b34-c565-51ae-8aac-b12ebea3f692","task_key":"default--validation--41972","task_revision_id":"1","upstream_id":"41972","short_description":"HellaSwag validation 41972","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to camouflage a scar [title] Evaluate the scar. [step] Scars vary in size and severity depending on the original cause. Determining the quality of the scar can help camouflage it.\",\"endings\":\"['Scar identification techniques and treatments may require surgery to hide the scar. Make a list of realistic areas, or include them in your life.', 'In order to determine the quality of the scar you will need to make a difference in determining its severity and intensity. [substeps] If you care about a scar that is darker than the original, determine how dark it is.', '[substeps] Keloid scars are scars that raise above the skin and do not go away over time. They often occur after surgeries, but acne and minor irritations can also cause keloid scars.', 'Scars tend to be round, triangular, or deep. [substeps] Check scars for coloration or roughness.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}