{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8d3d8be1-4327-55a6-a1df-6264764a0ae6","task_key":"default--validation--12463","task_revision_id":"1","upstream_id":"12463","short_description":"HellaSwag validation 12463","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose the right swimsuit (guys ) [title] Determine how you intend to use the swimsuit. [step] If you are buying a suit for racing competitively, you'll need a different suit than you would if you're just swimming for fun. The more leg movement (i.e., actual swimming) you intend to do, the less fabric you should have on your swimsuit.\",\"endings\":\"['[title] Decide what sort of style you want to adopt. [step] If you want to really impress, think carefully about your build and research what sort of swimsuit looks best on someone with your body type.', \\\"[title] If you are wearing a bikini, make sure you have plenty of sunscreen. [step] Even if you use the sunscreen, you still need a little protection against the sun's rays.\\\", '[substeps] Continue to add extra fabric to your swimsuit just in case you reach the top limit. Remember to reduce, or decrease, fabric on your swimsuit.', \\\"[title] Pick a suit color that complements your skin tone. [step] For example, wearing a green suit can look great on all ages, but it's a shade harder for women's bodies.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}