{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1808a057-f063-55a3-aba4-8d21296a0964","task_key":"default--validation--7786","task_revision_id":"1","upstream_id":"7786","short_description":"HellaSwag validation 7786","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to decorate your home with pastel colors [title] Look for pastel accessories. [step] If you only want to add small touches of pastel to your home, then adding small accessories is the best way to do it. For example, in the kitchen you could put out a light blue butter dish.\",\"endings\":\"['On the living room wall you could add a purple canvas, or put a robin roost on the couch. Think of an exciting color, with comfort and aesthetic appeal that pleases you.', 'In the living room, look for light green coasters. In the bedroom, add a lampshade in a light pink or purple color.', 'On the dining table you can use a tray of pastel decorations. [substeps] Sunflowers, daisies, and other herbs such as sprigs of rosemary or basil all work well.', \\\"Or in the living room you could have a tote filled with notes and a lamp. Open the tote and you're all set.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}