{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f3c453c-18dd-58ec-98c9-1d6537972c68","task_key":"default--validation--37607","task_revision_id":"1","upstream_id":"37607","short_description":"HellaSwag validation 37607","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to deal with a lying teenager [title] Let your teen know when you've caught him/her lying. [step] If you've caught your teen in a lie, you'll need to address the lie and the underlying behavior (whatever he/she was lying about). However, it's imperative that you do this as carefully as you can.\",\"endings\":\"['Tell your teen something like, \\\" max, i\\\\'ve caught you lying a lot before. I am so mad at you! \\\" after telling her lying, wait for her to solve the problem.', \\\"Don't just tell your teen they are lying, drop the question after the fact so you'll know what to address. A good rule of thumb is to intervene if your teen is lying to you the first time they call you.\\\", \\\"Otherwise your teen will just get upset back at you, and may be less likely to communicate with you about other things. [substeps] Don't act triumphant or satisfied by catching your teen in a lie.\\\", 'Remind your teen to thank you and explain why he/she did it. This keeps the problem between you and your teen from escalating into an argument or further conflict, or at least drawing suspicion.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}