{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3ba15479-749a-5e52-b6b4-ea74789c7b1e","task_key":"default--validation--22430","task_revision_id":"1","upstream_id":"22430","short_description":"HellaSwag validation 22430","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to help a pregnant teen friend [title] Explore her choices. [step] Talk with her about the options she has and her feelings about each of them. Talk about each option without judgement or pressure as to what she should choose.\",\"endings\":\"['[substeps] For example, if she wants to go to her doctor for an abortion and a friend wants to go, keep it brief. Reassure her that the decision is yours, not whether she should have to decide which one.', 'This is her body, her pregnancy, and she is going to be the one who has to deal with the consequences, not you. [substeps] She could decide to have an abortion.', 'This helps your friend learn if she would want to adopt a child, how to care for her baby, and just how the process works. [substeps] During this time, listen to your friend.', 'If you believe there are going to be alternatives to her choice, take her in for your talk. [substeps] Give her some suggestions on which options you think she might be more successful in.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}