{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75d806dc-9dc3-559c-9a73-41acf51a92ae","task_key":"default--validation--42663","task_revision_id":"1","upstream_id":"42663","short_description":"HellaSwag validation 42663","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to choose salad dressing [title] Browse through the aisles of a grocery store to see what types of salad dressings are being sold. [step] Here are some common types that you might find and their descriptions. [substeps] Caesar: contains oil and vinegar with a seasoning of romano cheese and garlic.\",\"endings\":\"['This is probably best used if you need to replace sodium. Sandwich: contains extra fat that you might not notice, but it is a good salad dressing.', 'Cole slaw: a sweet and creamy salad dressing that can be absorbed easily. French: there are 2 types of french dressings, namely separating and non-separating (non-separating is creamy whereas separating is more runny).', 'Bouillon: contains added capers containing salicylates and can be added to the dressing, if desired, or sprinkled over salads. The oil refers to the coating under the leaves of the lettuce, which you can add in whatever type of salad you like.', 'Caesar also has a strong broth-like taste and a rich earthy flavor, with a layer of cilantro in a thick dressing. Edamame: contains lactobacillus hydroxide in a thick dressing but a rich earthy flavor.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}