{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c7bb871-73ab-52e2-8802-be00a4e3978b","task_key":"default--validation--45396","task_revision_id":"1","upstream_id":"45396","short_description":"HellaSwag validation 45396","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to handle challenges with a troubled spouse [title] Choose the right location. [step] If your conversations tend to turn into screaming matches, pick a public location to talk. The key is to pick somewhere that doesn't tolerate loud talking, or a place in which you'd be embarrassed if anyone heard you yelling.\",\"endings\":\"[\\\"[substeps] Many employers and the entertainment industry have office visit policies in place to monitor what confidential information your spouse shares with others. It's important to have an appropriate place to talk in private.\\\", \\\"Pick a place where there's low visibility, and you won't be interrupted by the other members of the household. [substeps] For example, if you feel you won't be heard much in a large public space (like a restaurant), consider choosing a relatively quiet street or parking lot.\\\", \\\"A public domain like a library or a parking lot will work, as long as it isn't an area prone to disorder or racism. [substeps] Avoid public places like restaurants, as they might draw attention to you and your relationship.\\\", 'By choosing the right spot, you may have a more productive conversation. [substeps] Libraries, coffee shops, busy parks, and even the shopping mall are all good places to have a potentially heated conversation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}