{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"782c5085-6432-5075-9e40-f7180025c7d1","task_key":"default--validation--50653","task_revision_id":"1","upstream_id":"50653","short_description":"HellaSwag validation 50653","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to buy a peridot [title] Look at a variety of stones. [step] Expect peridot to come in various shades of green. Visit a respectable jeweler to browse a number of samples, all at once.\",\"endings\":\"['You will want to watch several of the gemstones, particularly emeralds, before you make your final selection. [title] Wear a pair of non-peridot shoes.', 'Not only are they among the delicates among them, but they can be broken down into shades of green. Try different sizes, brands, colors, and textures to create a base purchase.', 'Familiarize yourself with the different shades that it comes in, from olive to rich green to slightly yellowed. [substeps] The iron content in each gem determines its color.', 'Neither peridot nor many other jade or allekite stones are necessarily bad for you if you are only buying one or two. Make sure you trust your first look at a stone before buying one.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}