{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0b0ddb87-1eec-561b-bc06-30d22d3ed652","task_key":"default--validation--34138","task_revision_id":"1","upstream_id":"34138","short_description":"HellaSwag validation 34138","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to trim down the size of your wedding guest list [title] Determine your budget. [step] Have a conversation with your significant other and share your wedding goals, needs, and wants. Once you have shared these, come to an agreement on how much each of you can feasibly contribute, including possible contributions from your families.\",\"endings\":\"[\\\"[substeps] Budget for the entire wedding can be complicated, and the costs should be discussed in advance. If you don't know exactly how much your wedding guest guest needs to contribute, bring together a few friends and family and ask them together to help plan an event.\\\", \\\"[substeps] If your money is more than what you need in order to attend the wedding, it's likely that you cannot contribute all of it. Consider asking family and friends to look over your guest list.\\\", '[substeps] In other words, rather than hiring an attorney to draft your list of possible contributions to your wedding party, consider hiring an accountant to draft a draft. You may also meet with a family member or significant other to discuss how much this person will be making each month.', \\\"Decide on who will pay for what, and commit to this agreement with your future spouse. [title] Identify your venue's maximum capacity.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}