{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f25218d3-0166-53e5-89d0-dba9dd3389af","task_key":"default--validation--40609","task_revision_id":"1","upstream_id":"40609","short_description":"HellaSwag validation 40609","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to curl your eyelashes [title] Open the curler and hold it at the base of your lashes. [step] Take the eyelash curler in your dominant hand, and move it to the open position. Hold it up to your eye, placing it so it's at the base of your lashes.\",\"endings\":\"[\\\"[substeps] Also start above your eyelashes while you are holding the curler. It's important that you hold it at the base of your lashes as well.\\\", 'Gently release the curler and roll it upward over the lash line. Continue rolling the curler downward until the clamp clamps down on the eyelash.', \\\"[substeps] Don't apply mascara to your lashes before using the eyelash curler. The lashes can stick to the curler's pads if they have makeup on them and you may wind up pulling them out when you curling.\\\", 'Let the curler slide down your eyelashes. Wrap the other end around your lashes to create a match shade.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}