{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8d599e90-9ef4-5dd6-9c3e-354be96101f9","task_key":"default--validation--4859","task_revision_id":"1","upstream_id":"4859","short_description":"HellaSwag validation 4859","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to rent your bridesmaid dresses [title] Determine your budget. [step] Before renting a bridesmaids dress, think about your personal budget. If the wedding party is paying for dresses, they should be able to give you a rough budget of how much to spend.\",\"endings\":\"['Gather all of your spending money to calculate your wedding budget. Research wedding dresses available to you online to save time and save yourself if your budget takes over.', \\\"If you're paying for the dress yourself, think about your personal budget. [substeps] Think about how much income and expenses you have each month and how much disposable income you have.\\\", \\\"Decide how much you're willing to spend to save. [title] Research the bride and groom's input.\\\", '[substeps] You may also be able to find a dress online that has a reasonable cost. Checking local wedding shops may allow you to find wedding dresses for free for a more reasonable cost.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}