{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"431a9a86-32a0-5ba7-a97e-1efbc94b6350","task_key":"default--validation--3852","task_revision_id":"1","upstream_id":"3852","short_description":"HellaSwag validation 3852","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to break in a new pair of shoes [title] Wear your new shoes around the house. [step] Before you go out in them, try walking up stairs, standing around (cooking dinner, playing with kids, etc. ), sitting down, and even.\",\"endings\":\"[\\\"It is vital to have them or at least make sure they're there. Here are some other things to try : [substeps] Get rid of pencil marks when they pop up.\\\", '[substeps] This is the most trusted method for easily and lightly breaking in your new shoes. If you have nice leather or dress shoes-shoes that you would be frustrated to see scuffed, altered, or even discolored-this method is the safest to try.', 'A lot of people are intimidated by the idea of wearing them around the house, just to say, \\\" i look cool at my new pair of shoes today \\\". However, keep in mind that these pairs of shoes not only make you more confident in your appearance, but they tend to make your lower body look properly toned and toned.', \\\"After you're ready to put on your new pair of shoes, put them on your feet and you can go up each time. It may help to fill in some space with a simple bowl of soup.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}