{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"522cef4d-959f-5537-a16c-9a7c582e264e","task_key":"default--validation--31412","task_revision_id":"1","upstream_id":"31412","short_description":"HellaSwag validation 31412","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to react when a companion spends the entire time on their phone [title] Point out the issue you're seeing. [step] Identify what your friend or partner is doing and let them know how it makes you feel. It is easy to feel snubbed if your companion checks their phone rather than engaging with you.\",\"endings\":\"[\\\"Without even recognizing it, they may receive unwanted notes or notifications, which might lead to aggression. [substeps] If your friend has a cell phone, tell them you'll be back by calling or emailing them shortly.\\\", 'They may be doing it just to take away your jealous feelings. [substeps] For example, say something like, \\\" when you spend the whole time texting me, it makes me feel really annoyed.', 'That said, they may not even realize they are doing it. Give your companion an opportunity to realize their mistake.', 'This will diminish or even eliminate their presence in social situations. [substeps] Say something like, \\\" rachel, i\\\\'m very busy right now and would really appreciate it if you hadn\\\\'t drifted off on twitter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}