{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f4dc085-1f1f-5058-95bb-54c05a9eea7b","task_key":"default--validation--25268--variant-6606f60cdd00","task_revision_id":"1","upstream_id":"25268--variant-6606f60cdd00","short_description":"HellaSwag validation 25268","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to cook pork tenderloin [title] Create a dry rub. [step] Using your favorite mix of herbs and spices, create a \\\" rub \\\" to spread over the uncooked tenderloin. The flavors will form a crust when the tenderloin is cooked.\",\"endings\":\"['[substeps] The spices will act like a thick syrup to aid in cooking. It will then absorb the hot water and add to the ham.', '[title] Add the chicken to the broiler. [step] When your oven is preheated, add your chicken to the broiler.', \\\"[substeps] You'll need about ½ cup (48 g) of dry rub for every pound of pork tenderloin. Simply spoon the rub over the tenderloin and use your hands to spread it so it covers the entire cut of meat.\\\", '[substeps] Use your favorite flavors of herbs and spices to create a rub. However, you can also make a rub for pork tenderloin with seasoned beef and ground beef as well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}