{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"025a19e7-6af2-5c58-93b8-c6f92204618c","task_key":"default--validation--21820","task_revision_id":"1","upstream_id":"21820","short_description":"HellaSwag validation 21820","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make pork and herb breakfast sausages [title] Buy, borrow, or otherwise finagle a portable meat grinder. [step] There are a number of types available, here are a couple. [substeps] Hand cranked grinders.\",\"endings\":\"['These are small handheld grinders, often stainless steel. These will allow you to deep quarter and remove a pound of meat from a single skillet.', 'Wider diameter grinders are ideal for cutting up pungent and high fat sausage. These grinders hold different amounts of fat in equal parts.', \\\"These are the cheapest, and are usually pretty sturdy apparatuses, but they require the user to turn the crank, which in turn spins an auger that feeds the meat into the cutter wheel. They commonly clamp to a table top, to free the user's hands up for other parts of the process.\\\", 'One of the cheapest and inexpensive and are available in a lot of different sizes and shapes. They are a bit like sausages but very efficient, reliable, and are cheaper.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}