{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0918c8f5-df49-50ac-8e24-621c1aec572b","task_key":"default--validation--33682","task_revision_id":"1","upstream_id":"33682","short_description":"HellaSwag validation 33682","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make a homemade diaper [title] Use a shirt that is 100% cotton. [step] Cotton is more absorbent than most synthetic fibers, so it makes a better material to use for a cloth diaper. [substeps] Use a short sleeve or three-quarter-length sleeve for best results.\",\"endings\":\"['A three-quarter length sleeve may make it easier to pin the diaper on larger babies and toddlers, but may be more material than you need for smaller babies. Choose a size based on the size of your child.', \\\"You don't need to wash the shirt before wearing it. If you do wash your shirt before putting it on, then don't roll up the sleeves so you don't have to lift the shirt down when pulling it on.\\\", 'Look for a cotton with only one color on it. Darker colors will make the fabric more prone to wrinkling.', '[title] Measure the size of your chosen cloth diaper from head to ankle. [step] Purchase this measurement at a home improvement store.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}