{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86f1f445-9d26-5255-b25c-2562b4513bea","task_key":"default--validation--24326","task_revision_id":"1","upstream_id":"24326","short_description":"HellaSwag validation 24326","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to tell when your parent is sad [title] Monitor any changes to their normal behavior. [step] Sadness will often cause some subtle changes in the way your parent acts. For instance, they might normally be very talkative, but lately they have been quiet.\",\"endings\":\"['Pay attention to any marked changes in their usual behavior or routines. [substeps] Consider whether they are ruminating on an event or conversation.', \\\"Now is an ok time to take a look at what's going on. [substeps] Instead of feeling bad, listen for any shifts to your parent's demeanor that may indicate sadness.\\\", \\\"And if they suddenly suddenly start doing everything else, then maybe they don't like to do it anymore. Understanding the change may help you a lot in identifying the cause.\\\", 'Their chatter can become so abrupt, and they might say, \\\" oh my gosh, mom, everything would look so unfair. \\\" listening to this kind of \\\" sweet talk \\\" with your parents will alert you when a change is taking place.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}