{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"30c9aa82-97a5-5269-b389-a3f7305f969f","task_key":"default--validation--44676","task_revision_id":"1","upstream_id":"44676","short_description":"HellaSwag validation 44676","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to create a wedding guest list [title] Determine a ballpark figure. [step] Even if you're not sure on the specific figure, you can determine roughly what size of wedding you want. This will help you get quotes from caterers and venues while allowing you to figure out the specific number later.\",\"endings\":\"['[substeps] Fancy the style of wedding gowns you choose based on your personal personality. For example, a strapless dress with a pear-shaped crème de la ceci is a classic.', 'You can do this by reading wedding magazines and watching events on television. [substeps] Ask for key information, such as tables, seating and centerpieces, so you can take note of all the details for each venue, if applicable.', '[substeps] Some people decide just to invite immediate family members and a few close friends. A small wedding is generally around 50-75 people.', \\\"[substeps] If you don't have a ballpark figure, think about the group of people attending the wedding. If you choose to only host guests, consider hundreds or thousands of people.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}