{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac378db3-f7a2-5d1e-b1cd-de7357889243","task_key":"default--validation--49124","task_revision_id":"1","upstream_id":"49124","short_description":"HellaSwag validation 49124","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to function in a multigenerational household [title] Know why this arrangement is in place. [step] Every living arrangement has its benefits and drawbacks, and in order for it to work, the benefits must be recognized and outweigh the drawbacks. When the going gets tough, remembering why you've chosen to live in a multigenerational household will help to cope with any conflicts that may arise.\",\"endings\":\"['Some of the reasons families choose to live together are : [substeps] Money. Loss of a job or unexpected expenses, like divorce or death in the family.', \\\"[substeps] Five quarters of your population can still live in a one-bedroom household, though it's easier to accommodate one less person. [title] Learn what makes a modern lifestyle unique.\\\", '[substeps] The primary disadvantages of it are that not sharing a household room is illegal. It might be good for everyone to have a family room as well as a shared living room.', 'It will also help that you are able to get along with your neighbors, as your personal boundaries will be met as well. [substeps] Your family will not be on your side unless you agree.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}