{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"16eebf73-eee9-575c-8a8d-f73ed13f6b29","task_key":"default--validation--17024","task_revision_id":"1","upstream_id":"17024","short_description":"HellaSwag validation 17024","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to shape a cowboy hat [title] Decide on the brim shape you want. [step] Many felt cowboy hats are made with flat brims, so you can shape your own to the style you prefer. You can shape your hat based on aesthetics alone, to complement the shape of your face.\",\"endings\":\"['Use your pencil marks to draw the brim shapes onto a piece of felt cloth. [substeps] If you want a hat that has round side corners, create the brim shapes on the cloth by clipping on the sides of the brim shape shapes.', 'A rule of thumb is that the narrower your face is, the higher you should bend the edges of your hat. [substeps] If you have a rounder face, the edges should not be folded up far.', 'Adjust the brim shape throughout to create the desired look, then spend some time adding more felt to define the brim. [substeps] Make sure you are satisfied with the appearance.', '[substeps] For example, if you have an oval face, then your brim will be an oval shape. Or, if you have a head shaped head, then your brim will be round.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}