{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26a099e7-9b1e-58c5-88c7-49f713c815cf","task_key":"default--validation--18551","task_revision_id":"1","upstream_id":"18551","short_description":"HellaSwag validation 18551","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to break in high heel shoes [title] Wear them throughout the day. [step] The first step to breaking in a new pair of high heel shoes is simply to wear them often. The more often you wear your heels, the more of an opportunity your heels have to stretch and conform to your feet.\",\"endings\":\"['[substeps] To avoid the awkwardness and potential danger of wearing new high heel shoes while out and about, start by wearing your shoes at home. You can wear them while watching television or folding laundry.', \\\"Your heels should be comfortable to wear and properly worn throughout the day, and it's likely that if you wear your heels too frequently, your heel will eventually rub against whatever area of your foot would end up hurting. [substeps] Both high heels and low heels can cause soreness in a variety of different areas of your feet.\\\", 'If you wear them too often, your flip flops may break and slide off of your foot. [title] Consider walking shoes before you actually wear your new heels.', 'You can go out and buy some high heels at your local store or choose a pair you know you will be comfortable wearing and wear them for a comfortable time. [substeps] The first step to getting into high heels is simply to wear them throughout the day.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}