{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"19711bad-aaff-5997-9ee2-4dd1649919f7","task_key":"default--validation--45657","task_revision_id":"1","upstream_id":"45657","short_description":"HellaSwag validation 45657","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a pin [title] Add a pin to your collar. [step] Putting a pin, or several pins, along the neckline of your dress or blouse can add pizzaz to your look. If you have one pin, place it in the center of the neckline.\",\"endings\":\"['A pin with buttonholes or a small rhinestone, such as a stencil or rhinestone, can pop out perfectly. [substeps] Keep these pins in a safe place, such as between your bra and underwear.', '[substeps] For example, if you have a jacket with a colorful wrap, place a pin onto it. You can wrap it around the inside edge of the jacket and pin it in place.', 'If you have several pins, place them at the top, at the armpit or corner of the collar. Draw a comma around this pin, to serve as a base or extra brace.', 'Alternatively, you can place the pin, or pins, off-center. Be sure to space them evenly and balance out the sizes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}