{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40deb11e-0a4f-5027-a893-fc2071d637ab","task_key":"default--validation--35656","task_revision_id":"1","upstream_id":"35656","short_description":"HellaSwag validation 35656","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be a recluse [title] Prepare yourself for some fear. [step] Spending time alone is scary for many people. In our modern era, we're more interconnected than ever.\",\"endings\":\"[\\\"When we're alone it's one of the main fears something might bring up. Living in a recluse for very long can distort our sense of well-being.\\\", \\\"Give yourself a break if you can't handle too much social interaction. [substeps] Just let one person into your house, lock the door, turn your lights off, and stay for five or ten minutes.\\\", 'If you want to take some time to be reclusive, there will be an adjustment period. You may feel scared or confused when you first begin spending time alone.', 'Despite hardships and extraordinary life events, we could easily become lonely and isolated in our homes. In these situations, you will know that you can come out and accomplish a goal with no fear of being alone.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}