{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"71d30b13-c039-54fd-8548-ec234ffeaabc","task_key":"default--validation--20008","task_revision_id":"1","upstream_id":"20008","short_description":"HellaSwag validation 20008","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a brooch [title] Purchase felting wool in a variety of colors. [step] It can be found online or at any number of brick and mortar craft stores. It's pretty cheap, so buy a few kinds--you'll like the variegation in your end piece.\",\"endings\":\"[\\\"Felting wool actually kind of looks like fleece, but for your brooch, you'll need to pull it out. Some department stores also carry felting wool, which you can wrap around your brooch at a cheaper rate than synthetic fabrics.\\\", '[substeps] Felted wool is not regular wool. It will come in soft, puffy bunches.', '[substeps] Felting wool cloth is also fairly inexpensive. Thick wool is easier for collectors to recognize.', '[substeps] Felting wool is most often used as plastic cutlery, but it can also be used to create little wreaths and fabric flowers. You will want to collect the wool from a local junkyard.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}