{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ffb253f4-7f1d-5d80-aac3-5655025f4dcd","task_key":"default--validation--26380","task_revision_id":"1","upstream_id":"26380","short_description":"HellaSwag validation 26380","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to evaluate loose diamonds [title] Buying that perfect loose diamond is not difficult at all, especially if you know what you are looking for. [step] The best way to make a conclusion about the cost of a diamond-is to understand what makes it desirable. Four major criteria define the quality and value of the diamond-its carat weight, diamond's cut its clarity and its color.\",\"endings\":\"['As a general rule, diamonds should be sold by a jeweler, who will inspect the stones closely. Children, teens, and older persons are also likely to purchase off-the-shelf, expensive, timeless pieces.', '[substeps] Flex diamonds are probably the most economical way to calculate mass, while gold are a good choice. Mm diamonds cost less than mm diamonds, but you will generally save money on more precise measurements.', '[title] Determine the weight of the loose diamond. [step] A diamond is normally weighed in carats.', 'You will find diamonds that do not reflect these criteria at all due to constant wear and tear. Because of this, diamonds that were costly during the years of cosmetic do not reflect their worth or durability.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}