{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ba5b0d8-4512-5d4b-bed7-b50eebd48544","task_key":"default--validation--258","task_revision_id":"1","upstream_id":"258","short_description":"HellaSwag validation 258","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to break in your brand new dr martens boots [title] Start with thick socks. [step] Put on socks that will both push the leather outwards and protect your feet from any rubbing or pinching, preventing the formation of blisters. [title] Wear the shoes for 10 minutes.\",\"endings\":\"[\\\"[step] Once you've put them on, stand up and observe the look. You will notice that the heels slightly raise and the toes stick slightly upwards.\\\", '[step] If they are not waterproof, soaking them in water for 5 minutes every half hour are enough for optimal dryness. [title] Take them off and gently clean them.', '[step] Keep the shoes on long enough that they begin to mold to your feet, but not so long that they cause pain or injury. Walk around and stretch your feet while your wear them.', '[step] Otherwise, you may accidentally put too much of your foot on the boots. [title] Remove the boots and turn them inside out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}