{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03f482ca-2f69-5b99-814c-0e4d315f8147","task_key":"default--validation--45597","task_revision_id":"1","upstream_id":"45597","short_description":"HellaSwag validation 45597","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to eat in amsterdam on the cheap [title] Buy french fries for a snack. [step] Known as \\\" vlaamse (flemish) frites \\\" in the netherlands, and most often served with mayonnaise, french fries are sold everywhere in amsterdam from restaurants to street stands for a very reasonable price. Fries come in three or four different sizes, from small to meal-sized large, to satisfy any appetite.\",\"endings\":\"['[title] Grab an èsan de vouches. [step] Handmade pre-made french fries are best used as a snack, as they are made of basic ingredients rather than other ingredients.', '[substeps] Optional sauces are available for an extra fee. These might include the most popular mayonnaise, as well as curry sauce, garlic sauce, gravy, cheese, and ketchup.', 'If you are eager to please, buy french fries in seven to nine batches. The larger the frites, the larger your bag going.', 'Remember to buy some pepper on the side, too. [substeps] You should try to purchase about one inch of french fries, or else the greasy fat might turn your plate green.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}