{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18613374-53bb-59a9-a009-0f289488697f","task_key":"default--validation--2589","task_revision_id":"1","upstream_id":"2589","short_description":"HellaSwag validation 2589","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear lace up pants [title] Select the most flattering waist for your body shape. [step] When shopping for lace-up pants, you will typically have a choice between low-rise, mid-rise and high-rise waists. Make your choice based on the style that looks the best with your body shape and what you feel most comfortable in.\",\"endings\":\"['[substeps] Dresses and dresses with ruffles or fringe can help add volume to your figure. Choose a dress or blouse with ruffles to make your legs look as shapely as possible.', 'For example, high-waisted pants tend to flatter most body shapes. Choose mid-rise if you want to play down the size of your lower half.', 'High-rise will look best for a slimmer shape, while low-rise will look best for a more skinny or slender figure. [substeps] To determine your measurements, start with an allowance of around half an inch or less.', 'A waist that is noticeable even at the top or bottom is not suitable for clothing that is too low. [substeps] Tight jeans in a neutral material tend to create too much bulk.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}