{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eea3eb09-0740-5ee8-9cf5-dceddd4ca1aa","task_key":"default--validation--47047","task_revision_id":"1","upstream_id":"47047","short_description":"HellaSwag validation 47047","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to tie a turban [title] Fold the fabric. [step] Fold the page over four times length-wise, making sure that the ends are aligned. Ideally, your fabric should be about 6 yards (or larhs) long, so that you have enough room to fully wrap the fabric around your head.\",\"endings\":\"[\\\"When you're left with one half of the turban, fold that half over twice and tie a knot at the right or left edge. This will keep your turban from rubbing against your head.\\\", 'The fabric you start with should be made of cotton and should be as thin as possible. When you fold it four times, it should be about 2 inches (5 cm) wide.', 'The folded fabric should rest securely around the inside of your turban. [substeps] If desired, you could use a decorative bow tie in which the two ends are tied together.', '[title] Pin the edge of the cloth to the length of the turban you are using. [step] Double check to make sure that the folded edge is going through the sashing, if necessary.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}