{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f0b8caf1-e51d-57c7-a5c0-c027c03ba97d","task_key":"default--validation--23850","task_revision_id":"1","upstream_id":"23850","short_description":"HellaSwag validation 23850","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to paint acrylic on canvas [title] Pick canvasses with a weight of 8 to 10 oz (230 to 280 g) per 1 yard (0.91 m). [step] A heavier weight indicates a better quality canvas. Look at the back of the canvas.\",\"endings\":\"['The darker it is, the better the canvas. The weight is determined by thread density, which also affects the texture.', 'Paint resistances read: \\\" these are horseshoe or sphinx-like bodies with muscled muscles under light skins that are white and smooth. \\\" [title] Use acrylic paint with very fine bristles.', \\\"It should not have to be flat so the paint doesn't appear to be frozen. Although acrylic paints might be a little waxy on the outside, they look and feel warmer on the inside.\\\", 'A flat back with holes in it makes it hard for paint to slide in and out. Take note of how much paint is in each hole by looking at the front of the canvas.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}