{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e262c635-48af-5d94-a666-1c2eceade8fb","task_key":"default--validation--29007","task_revision_id":"1","upstream_id":"29007","short_description":"HellaSwag validation 29007","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make a birdcage veil [title] Cut an oval out of buckram. [step] The oval should be no more than 4 inches (10 cm) long and 2 inches (5 cm) wide. [substeps] Buckram is a sturdy material used to form hats.\",\"endings\":\"['Cutting it to this size will help prevent it from rotting during its reconstruction. It will also allow for the adornment of feathers that will be in the hat to wrap around the wire to mimic the shape of a bird.', 'It works much better for small men as compared to thick ropes made from hard woods. [title] Fold the folded oval down so that it forms a long, rectangular frame.', 'It is not grassy and does not adhere to wet and slippery surfaces. [title] Fold down an end of the oval.', 'If you cannot find buckram, you could substitute another stiff fabric. Two layers of canvas or cotton lined with interfacing may work.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}