{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51ec4ee4-c76d-5702-a10c-1b062ca9cf46","task_key":"default--validation--1354","task_revision_id":"1","upstream_id":"1354","short_description":"HellaSwag validation 1354","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a gardener's apron [title] Cut a large rectangle for the apron base. [step] It needs to be 24 inches (60.96 centimeters) wide and 19½ inches (49.53 centimeters) tall. You can use plain or patterned cotton canvas fabric.\",\"endings\":\"['[title] Cut off the excess wrap at the base of the apron base. [step] Do not cut the excess wrap off completely; cut a small, horizontal tear in the fabric and sew it closed with another, slightly larger tear.', 'Make sure that the seam running the length of the fabric on both sides is flush with the rest of the fabric, otherwise it might wrinkle. [title] Cut the top half of the fabric inside out.', \\\"If you are using patterned canvas, however, be aware that both the right and wrong sides will be visible on the front of the apron. [substeps] You can also find a pattern online to suit your needs and style, if you're not comfortable designing your own apron.\\\", '[title] Fold each end of the long rectangle in half. [step] Unfold one side of the long rectangle to reveal the thick edge of the apron base.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}