{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28d1a450-8f1d-5876-98c4-a62e5cc3f977","task_key":"default--validation--30814","task_revision_id":"1","upstream_id":"30814","short_description":"HellaSwag validation 30814","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear high heels without pain [title] Get your feet sized at a shoe store. [step] Measure the length and width of your feet at a shoe store, and ask for assistance to find the most accurate size of shoe to buy. [substeps] Make sure to measure both feet.\",\"endings\":\"['One foot should be the largest right now; the other should be the smallest left. Also, make sure that you have a foot stretcher before the shoe shopping.', 'Click here for detail: http: // www.amazon.com/the left arch size is 7 inches (17. 2 cm), and the right arch size is 7 inches (17.', \\\"It's common to have slightly different sized feet. You may actually need to get two different sized shoes, or make sure to get the size of the larger foot.\\\", 'Standard heels typically carry a 7-inch to 9-inch (22.86-cm to 7-inch) heel on each foot. [title] Get your knees and heels size adjusted.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}