{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"31d8d216-b64f-58d4-bdf4-e8c64ee771fa","task_key":"default--validation--12460","task_revision_id":"1","upstream_id":"12460","short_description":"HellaSwag validation 12460","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to be a daring girl [title] Face your fears. [step] If you feel afraid of something, it is important to face your fears rather than running from them. For example, if you are afraid of heights, then avoiding heights will only intensify your fear.\",\"endings\":\"['Even if you want to keep your window up, avoid treading water all the time. [substeps] Know where your fear is coming from, but be willing to confront it head-on.', 'Instead, try to expose yourself to heights gradually to help reduce your fear. [substeps] You can also try making a list of things that you are scared to do from least to most scary, and try to do one every day.', 'Making a lot of noise, going up stairs, or talking to strangers can all cause anxiety. [substeps] Go somewhere loud like a park, school, or community center.', '[substeps] Tell yourself that you are brave enough to climb to the top. Take the jump, and hit the climb button.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}