{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8fee336e-5b9f-5495-98ac-15a6696d2055","task_key":"default--validation--29446","task_revision_id":"1","upstream_id":"29446","short_description":"HellaSwag validation 29446","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to look good in front of your crush [title] Make sure you have a friend or two close by. [step] Avoid one-on-one encounters with your crush until you've practiced getting your nerves under control. Hanging around in a group setting will take a lot of pressure off of you and make you feel a lot less self-conscious.\",\"endings\":\"['[substeps] If your friend invites you over for coffee or to a movie, go with her. Many people like to have other opportunities to spend time with their friends.', '[title] Be ready for your meet-up! [step] Visit your closet to find inspiration. Find sweatpants, leggings, or any other loose fitting clothing that will make your legs twitch.', \\\"Even if they don't know about your crush, your friends being there will lend the moral support you'll need to relax. [title] Try dressing up a bit if you think you may run into your crush.\\\", '[substeps] If you do see him or her standing with your friends and trying to get your attention, move away from the two of them. If he or she is with someone else, simply ignore them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}