{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f0e1febe-e9f1-5095-a5e2-b1f0cdf95be4","task_key":"default--validation--48031","task_revision_id":"1","upstream_id":"48031","short_description":"HellaSwag validation 48031","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to shape eyebrows before waxing [title] Mark where your eyebrows should begin. [step] Start by marking where you want your eyebrow to begin with a white pencil. The \\\" beginning \\\" of your eyebrow is the inside edge.\",\"endings\":\"['If you want the shape to start about ½ inch (1.3 cm) away from where your eyebrow would start, use a different pencil. Start at the bottom corner, and work your way across the other corner until you see the desired shape.', 'Using the white pencil will make it easier to see the shape of your eyebrow when you go to wax. [substeps] For instance, a good place for your eyebrow to \\\" start \\\" from the inside is in a straight line with the inside of your nostril.', 'This outer edge will become the inner edge of your eyebrow. [substeps] White pencil can be ordered online or from your local pharmacy, and may also come in both inexpensive and specialty sizes.', 'Look at the outside edge of your eyebrow as well. Is your eyebrow going inside or outside? [substeps] Draw a circle at this inner edge.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}