{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a32e52a8-07b8-50ad-aaf3-0784264d0a44","task_key":"default--validation--46839","task_revision_id":"1","upstream_id":"46839","short_description":"HellaSwag validation 46839","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to be a successful tutor [title] Introduce yourself to both the parent and student. [step] Make sure you are confident and welcoming, making both people feel comfortable with your services. It is important to communicate expectations and goals at the very beginning of your job, so that everyone's voice is being heard.\",\"endings\":\"[\\\"Moreover, communication is the key to your success as a tutor. Make sure you're up front and always show that you mean business.\\\", '[substeps] For example, you could introduce the person you are tutoring by saying something like, \\\" i\\\\'m william johnson. I am looking for a tutor to help me manage my monthly bills.', 'Allow the child and parent to have a discussion with you about the areas in which the child experiences struggles or successes. For example, they may have a fantastic vocabulary, but have difficulty with sentence structure.', '[substeps] Focus on establishing relationships with both parents and the student. Good communication skills will help the parent and student navigate the confusing span of time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}