{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"71c71d66-2fe3-55cd-88b2-883dbea00ed2","task_key":"default--validation--23981","task_revision_id":"1","upstream_id":"23981","short_description":"HellaSwag validation 23981","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do chrome eyes [title] Put eye shadow primer on your lids. [step] Makeup primer is a good foundation for any eye shadow look, but it's especially important when you're using bold shadows. Nothing will ruin your chrome eye makeup more than it creasing or smudging all over your face.\",\"endings\":\"['Sweep around the shade of your eye liner, replacing it with a darker shade. This will keep the liquid of the primer from globoning up onto your lid tissue.', 'Put a thin, opalescent primer onto your lids and use your fingertips to blend it in. [substeps] To get the smoothest, darkest application, go with your favorite shade.', 'A primer will smooth out and perfect your skin while also giving your makeup a sticky base to adhere to. It will help your makeup stay put and last all day and night.', 'Apply the primer on your lids, starting at the outer corner of your eye, and working your way towards the inner corner. It should stick like glue! [substeps] Use a cotton swab to check for any unevenness.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}