{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c8b7930-db2c-5b6f-ac89-9a52899f3b9d","task_key":"default--validation--13511--variant-59a524a140b9","task_revision_id":"1","upstream_id":"13511--variant-59a524a140b9","short_description":"HellaSwag validation 13511","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to locate a baby in infant adoption [title] Identify the appropriate agency in your state. [step] Your state will have a children's services or human services agency that handles foster care and adoptions. Using a state agency typically is cheaper than other methods, but may have drawbacks.\",\"endings\":\"[\\\"[substeps] Typically you'll need to file an application for a child adoption online. You'll need to provide your personal information, such as the name of the baby you're seeking, as well as all the addresses for the foster parents you plan to foster.\\\", '[substeps] For example, the agency might reject certain promises, apply for a grant, or sign off on a child in your home. It may be difficult to identify the agency just because you searched the database.', \\\"[substeps] For example, in many states you can only use a public agency if you are able and willing to adopt a child with special needs. Additionally, the vast majority of children available through the state's public agency tend to be older.\\\", \\\"[title] Get the name of the agency that best matches your needs. [step] You typically can find the agency name on the agency's website.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}