{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"baa70515-5335-5834-9579-65d70ec58460","task_key":"default--validation--10306","task_revision_id":"1","upstream_id":"10306","short_description":"HellaSwag validation 10306","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to hide the fact that you forgot to wear underwear [title] Don't panic. [step] Most likely nothing will happen and your lack of underwear won't be noticed by anyone because your clothes are covering your behind. You'll be just fine; the main thing is to be alert to anything that might happen if there is a gust, playing sports or getting something caught on your clothing.\",\"endings\":\"[\\\"[title] Don't wear your underwear when you do. [step] It's like hiding a panty that springs forward a crack at night when your partner is sleeping.\\\", \\\"Remember that you know what to do and do what you said you would do. [title] If needed, give yourself a' listen to' break.\\\", \\\"While you can hide your underwear in plain sight, you can also just wear them casually without being noticed by anyone at school. If you're comfortable wearing underwear, then you can wear whatever you please as long as nobody notices.\\\", '[substeps] You could talk to someone in the school office, if you feel comfortable. They may have spare clean pairs of new underwear for just such a problem.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}