{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5e29704-725d-55c0-86c5-5b685a9f0cb5","task_key":"default--validation--38665","task_revision_id":"1","upstream_id":"38665","short_description":"HellaSwag validation 38665","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to oven cook sausages [title] Pull the sausages out of the fridge 20 minutes before cooking. [step] About 20 minutes before you plan to put them in the oven, take the sausages out of the fridge and set them on the counter. This will take the chill off of them so that they cook more evenly.\",\"endings\":\"['[substeps] If you may be able to taste sausages briefly, do this right before you add them to the oven. [title] Place the sausages in an omelet pan.', '[title] Preheat the oven to 350 ° f (177 ° c). [step] If your oven takes a long time to heat up, you may want to turn it on before pulling the sausages out of the fridge.', 'While cooking, stir everything in so that everything is evenly distributed throughout. [title] Place the sausages in an oven bag.', \\\"Make sure they're fully dried before putting them into the oven. [substeps] If you cooked the sausages on a baking sheet or pan, place them on it the night before you plan to cook them.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}