{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51eff127-2d2a-531e-9f47-a7341dd8fba7","task_key":"default--validation--21877","task_revision_id":"1","upstream_id":"21877","short_description":"HellaSwag validation 21877","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to achieve full, sexy lips without surgery [title] Start off with absolutely nothing on your lips. [step] Then, apply a bit of cheap lip balm or sorbolene lotion to your lips. Leave this on them for a few minutes.\",\"endings\":\"['Rinse it off in the shower to cool. [title] Start drinking a good amount of water.', '[title] Start with your lips in a light shade with no pigmentation, so your lips appear fuller and more defined. [title] Put a bit of clear eye cream on your lips to help your lips appear plumper and kissable, without any pigmentation.', '[title] Pick up an old toothbrush, and lightly scrub over your lips. [step] Repeat this whole process, only make sure you use a grainy exfoliator this time, instead of the cream.', 'Smile and when your lips soften and tighten up, dab concealer on your lips. Massage as much on your lips as you can.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}