{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4f2f875-ee41-5daf-9d38-2e5573abda34","task_key":"default--validation--43096","task_revision_id":"1","upstream_id":"43096","short_description":"HellaSwag validation 43096","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to film skateboarding [title] Buy a mini-dv camera. [step] Primarily, the most cost-effective and high-quality cameras that you can use easily are mini-dvs that record digitally to tape. The quality is high enough, the tapes are easy to store, and the cameras are widely available and cheap enough that it's usually the best option.\",\"endings\":\"['The only downside of the camera is that it can be expensive, which can lead to frustration when you try to film one done on your own. Alternatively, you may want to invest in an additional camera to automatically take or save money.', '[substeps] A small panasonic or sony camcorder usually runs between $300 and $500. Sony premiums are the most popular mini-dv cameras among skaters, useful for their portability and quality.', '[substeps] This kind of camera does not streamline a lot of the \\\" film production \\\" of any video production project. You\\\\'ll probably have to buy a shorter-range camera, or a high-end model.', \\\"[substeps] Compact cameras, including the moto nica 290hb and 2 cofb cameras, can also be purchased for under $15 for a fee. You'll also need a pedal assembly kit (i.e.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}