{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26f7db54-ef41-5dec-90fd-baf7a26f8ab9","task_key":"default--validation--18893","task_revision_id":"1","upstream_id":"18893","short_description":"HellaSwag validation 18893","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make loom bracelets [title] Think about the sort of colors you would like to wear. [step] Do you have blue eyes? Match your bracelet to your eyes with an alternating blue/green, or blue/yellow bracelet. Are you planning on wearing a red outfit? Make a predominantly red bracelet.\",\"endings\":\"['[title] Help a child visualize a bracelet. [step] One way to do this is by pulling a pair of markers and asking them to draw a simple line that is two or three feet long, then make an imaginary row from the edge of each marker to the edge of the bracelet.', \\\"Maybe you don't want to match at all. You are versatile and unique, so why not make your bracelets the same? [substeps] Many websites sell multiple different colors, from monochrome bands to multicolored, tie-dye, and glow in the dark.\\\", '[title] Find some scrap material for your loom. [step] Flour and cornmeal are both great choices.', '[title] Wear zig-zag patterns. [step] Cut your fabric into a pale blue pattern for one pattern, and tie it into a knot around the middle of the bracelet.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}