{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5fb5f195-3917-52fe-99db-ed95aa55c312","task_key":"default--validation--31052","task_revision_id":"1","upstream_id":"31052","short_description":"HellaSwag validation 31052","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to bedazzle shoes [title] Choose the shoes you want to bedazzle. [step] You can bedazzle just about any type of shoe, from wedges to heels, canvas sneakers to flip flops. Plain shoes without a lot of embellishments (at least on the area you want to bedazzle) will work the best.\",\"endings\":\"['[substeps] A pair of plain heels that are closed-toe will achieve a starburst effect that is beautiful and elegant. They still require some bling to make them more individualistic, but fairly neutral too.', 'News shoes will work the best, but you can use older ones as long as they are clean and in good condition. [substeps] Because of how quickly they show wear and tear, flip flops and canvas sneakers should be new.', \\\"If you want to take an unconventional approach, you could wear pointed shoes and a pretty flat, wedge-shaped toe. This is fun and adds some character to shoes, but isn't the sexiest shoe to wear to bed.\\\", '[title] Collect additional materials you will need to complete your bedazzle. [step] These might include a large blow dryer or hair dryer, a bucket, and anything else that suits your needs.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}