{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17e3face-50f6-5d3a-8565-dc377ca74e54","task_key":"default--validation--32035","task_revision_id":"1","upstream_id":"32035","short_description":"HellaSwag validation 32035","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to prepare oolong tea [title] Choose your tea set. [step] For most tea drinkers, a ceramic teapot and cup will provide a good experience. Alternatively, you can put an infuser in your cup and brew a single serving directly.\",\"endings\":\"['Being careful of overheating or yeast leaks, you can pour your brew in the direction of the tea pot, if you do not have one already, and stir it regularly. Instant coffee is an excellent alternative, but you may want to do a little experimenting and see what is best for your circumstances.', \\\"[substeps] Typically, you'll want to make sure the cup is made from conductive material, such as porous ceramic. If you're brewing a hot tea, you can make yourself a styrofoam cup or a glass teapot.\\\", \\\"[title] Measure the required portion of the tea per serving. [step] Should you prefer a thicker, fruity tea, you'll need one serving.\\\", \\\"You can use any infuser, but a large cup-shaped insert works well as it allows the leaves to expand. [substeps] If you're a serious tea drinker interested in learning the chinese tea ceremony, look for a yixing clay teapot.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}