{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5f3b74b-4cc8-5c06-9232-20c1d55d418e","task_key":"default--validation--29057","task_revision_id":"1","upstream_id":"29057","short_description":"HellaSwag validation 29057","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to punch a hole in a belt [title] Purchase a leather hole punch. [step] If you want the new belt hole to be as tidy and discreet as possible, a leather hole punch is by far the best tool. These typically cost less than $10 us at a craft or hobby store.\",\"endings\":\"['You will need, however, a screwdriver and a measuring tape to set up the hole. [title] Cut a hole in the belt base.', '[substeps] Bring along your belt when shopping, to compare the size of the punch with the size of the existing holes. The punching tool tip should fit tightly through the existing holes.', '[substeps] Look online for a leather hole punch option at sites like h&m. [title] Create a notch in the leather by running your finger down the seam.', \\\"[substeps] In general, a leather hole punch uses an open area of the belt to punch, such as inside the belt. It's sharp enough for you to punch and work through at any time, and you can easily punch holes straight through the belt with just one hand.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}