{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2af1dbd0-cb2b-538b-9c0d-93ca0593fb62","task_key":"default--validation--46459","task_revision_id":"1","upstream_id":"46459","short_description":"HellaSwag validation 46459","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to measure your waist to height ratio [title] Get the right supplies ready. [step] You need a few things in order to calculate your waist-to-height ratio. Having everything ready will make this process quick.\",\"endings\":\"[\\\"[substeps] The first thing you'll need is a tape measure. Get a non-elastic, cloth measuring tape.\\\", 'Luckily, many calculators are available online at many major retailers. Have enough goody bags and a measuring tape to accommodate everything you need.', \\\"You'll need a tape measure, ruler, measurement calculator, order printed paper, binder clips, measuring tape and a ruler with a round tip. [substeps] Always measure from your mid-chest up to your waist.\\\", \\\"If you don't have supplies, you could just buy some unused ones from a grocery store. You'll need : [substeps] A ruler or tape measure ruler sewing machine thread scissors paper clip sewing wire cardboard baking paper scissors pins pressure cooker (help ease tension) measuring tape measuring tape plastic or wooden clothespins.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}