{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3889d5ca-2e3a-527c-85aa-7543929bbed4","task_key":"default--validation--21102","task_revision_id":"1","upstream_id":"21102","short_description":"HellaSwag validation 21102","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to kiss a girl the first time in your room [title] Ask if she would like to go to your room. [step] This can be done in a number of ways, but the best and probably most effective would be ask her if she would like to see your room. Something along the lines of, \\\" hey want to go upstairs? My room is warmer/nicer/more comfortable than here.\",\"endings\":\"['\\\" if she seems against it, don\\\\'t push it or you\\\\'ll make her uncomfortable and you should avoid making someone uncomfortable. You could also try using some different lines.', 'Is that cool? \\\" if she says yes, see if you can join her in your room. [substeps] For an older girl, it wouldn\\\\'t be proper to ask her out-you could invite your significant other.', 'And maybe we can go here one time. Is that okay? \\\" should she answer no, guys don\\\\'t like waiting around until other people are around.', '\\\" or \\\" we\\\\'re going to my house for a fun date/sleepover. Do you want to come? I just wanted to make sure you are ok with me being there.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}