{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e34ef4ed-bab1-54bf-ae1f-f14758ce95e9","task_key":"default--validation--45158","task_revision_id":"1","upstream_id":"45158","short_description":"HellaSwag validation 45158","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to have aurora's personality from sleeping beauty [title] Be graceful. [step] Graceful meaning delicate with her movements, as if floating in the air. Aurora is always graceful with her movements.\",\"endings\":\"['[title] Aurora is a sweet girl like any other. [step] She is never to judge but never to behave like her sister/daughter: she goes to the comfort of her own room when she sleeps.', 'When strolling in the forest with her animal friends, she walks with such poise and has perfect posture. When she picks some berries, her arms are fluid-she never does any harsh movements, like a ballerina.', 'This means graceful regardless of whether she is walking or hopping, she is actually graceful when moving. Hitting the ground with an improperly balanced foot, or bouncing, is equally graceful.', 'If you want to complement her, wear her clothing and her makeup. You can also wear your make up in a feminine way.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}