{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"916afb37-db6e-58d4-8723-8de006b3d2bb","task_key":"default--validation--23491","task_revision_id":"1","upstream_id":"23491","short_description":"HellaSwag validation 23491","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to buy condoms discreetly [title] Relax and keep your cool. [step] Take a deep breath and remember that buying condoms is responsible and normal. You will probably feel like everyone is staring at you and the cashier is judging you.\",\"endings\":\"['In reality, people are not paying that much attention to you. You are not the first person who has ever come to the store to buy condoms.', 'Do not be offended by these feelings or try to be taken seriously. You are doing the right thing and you could also break your fast and gain some privacy.', \\\"Even if you're embarrassed, just relax and ignore the stares. [substeps] If your name is ann smith, you may also consider calling yourself dr.\\\", 'Practice deep breathing techniques on your very own and practice switching your body language to show the cashier how you feel. [title] Pay attention to the name of the package and the make and model number.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}