{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9472364-74c6-538a-b104-de634aa69ad1","task_key":"default--validation--44167","task_revision_id":"1","upstream_id":"44167","short_description":"HellaSwag validation 44167","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to read a screw thread callout [title] Read the numbers. [step] They will look something like one of these : [substeps] 4-40 x. 5 1/4-20 x 5/8 m3-0.50 x 10 [title] Interpret the first number. [step] The first number gives the major, or largest, diameter.\",\"endings\":\"['Next, determine the size of the screw thread you wish to use. Write these numbers down on several pieces of paper and trace around them.', 'It also tells you the larger diameter of a screw thread to create a series of polygons which share a small degree of overlap by a single number. [title] Separate the single digits in each string from each string.', '[substeps] In unified threads (measured in inches) there are numbered diameters 0 through 10, with 0 the smallest and 10 the largest. ( diameters 12 and 14 may also be found, but are usually on older equipment and needed for repairs or restorations.', 'However, an \\\" x \\\" will bring more light up on an empty needle, so use a tray or ice scraper to fill it to feel the thickness of the screw. The next number tells us that the machine is a small amount bigger or larger than what we used it for.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}