{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bcc997b8-8422-5bea-b113-5d2b5e6a4bff","task_key":"default--validation--9715","task_revision_id":"1","upstream_id":"9715","short_description":"HellaSwag validation 9715","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a clutch bag with a bow knot [title] Cut the fabric pieces as follows : [substeps] For the body of the bag: cut two pieces of fabric measuring 10 \\\" x 6 \\\". For the strap: cut one piece fabric measuring 14. 5 \\\" x 3 \\\". For the bow: cut two pieces of fabric measuring 11 \\\" x 5 \\\".\",\"endings\":\"['For the final strap: cut two pieces of fabric measuring 22 \\\" x 6 \\\". For the first strap: cut one piece of fabric measuring 24 \\\" x 10 \\\".', 'The other piece is used to double-knot a button. For the neck: cut two pieces of fabric measuring 24 \\\" x 8 \\\".', 'For the zipper: cut one piece fabric measuring 35 \\\" x 10 \\\". For the ribbon: cut two pieces of fabric measuring 24 \\\" x 10 \\\".', 'An additional bow piece: cut one piece of fabric measuring 2 \\\" x 5 \\\". Lining: cut two pieces of fabric measuring 10 \\\" x 6 \\\".']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}