{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d0d81b6-e413-5f22-8aee-edb0f0fc4c46","task_key":"default--validation--17568","task_revision_id":"1","upstream_id":"17568","short_description":"HellaSwag validation 17568","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a finger loop bracelet [title] Gather your supplies. [step] For this you will need 5 strands of string, each about 2.5 feet (0.8 m) long but they all must be the same length. [title] Fold all of the strings in half.\",\"endings\":\"['[step] Lay the long strings outside in a corner and the long ones inside out. This will help you do the favors together.', '[step] Draw an x in the middle of each strand. [title] Thread the string through a knot in the previous one.', '[step] You should fold them all in half separately and then later gather them together to avoid having them twisted together. [title] Once all the strings are gathered together, tie a regular knot at the top of all the strings.', '[step] The lengths you need can be anywhere from 10 yards (2.4 m) and 6 feet (1.8 m) long. [title] Connect the ends to separate the long end and the longer end to separately connect the shorter end to the shorter one.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}