{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"571c4222-7488-55ba-b1c6-f0d4d02f225c","task_key":"default--validation--19237","task_revision_id":"1","upstream_id":"19237","short_description":"HellaSwag validation 19237","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to attend family counseling [title] Have a talk. [step] Getting loved ones to attend family counseling together might take some effort and organization. Whether you're trying to convince your spouse, your parents, or your siblings, you'll first need to have a frank talk.\",\"endings\":\"[\\\"This will make your children feel more supported, as well as allowing them to understand what you're going through. [substeps] Make sure you're talking openly about what you're going through, but be prepared to speak about work or responsibilities.\\\", 'Talk to them several times a day, and encourage them to come on board if they need it. [substeps] Encourage your loved ones to call and ask questions, too.', 'Ask your loved ones to sit down so you can communicate your desire with them. [substeps] Let your loved ones know that you want to have an important talk, i.e.', '[substeps] Organize a family meeting where everyone can discuss your concerns. Set deadlines and dates for each discussion.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}