{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"739e723e-6ee1-5d77-8a99-43c89a70ec5c","task_key":"default--validation--14083","task_revision_id":"1","upstream_id":"14083","short_description":"HellaSwag validation 14083","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to set up an aerial yoga hammock [title] Gather the material on both ends of the fabric. [step] Lay the hammock flat on the ground and place your fingertips on the left and right edge of the fabric. Start to walk your fingers up the fabric so that both ends bunch up.\",\"endings\":\"[\\\"Once you've collected enough fabric, put it around the outside of the hammock on both sides. [substeps] Make sure the large portions of the hammock are about 2 to 3 inches (5.1 to 7.6 cm) in diameter, not bigger than your forearm.\\\", 'Finish by pulling the fabric together so that the hammock stays close to your body. The fabric should lie on the floor of the hammock.', \\\"The bottom hem should be wrapped around the inside edge of the fabric and the top hem should be wrapped around the outside edge of the fabric. You shouldn't disturb or stretch this layer, but you should secure it in place with pins or clips.\\\", \\\"[substeps] Instead of looking like a sheet, your hammock should look like a loose rope when you're done gathering the material on both ends. [title] Wrap the end of the fabric around your arm.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}