{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b2748c34-22b1-5478-898e-6d738719f8fb","task_key":"default--validation--6805","task_revision_id":"1","upstream_id":"6805","short_description":"HellaSwag validation 6805","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to survive financial hardship in a marriage [title] Remember your wedding vows ,' for better, for worse, for richer, for poorer'. [step] When we make these vows to each other it probably doesn't enter our minds what could happen down the road, but nonetheless we made a promise that we had the best intentions of keeping. During difficult financial times those vows become far more prominent and necessary.\",\"endings\":\"['[title] Remember not to shut doors at all hours. [step] Shut off all lights at any time, especially those that serve to light up your wedding day.', 'Stick with it not because of the promise you made but because of the person you made it to. [title] Remember that as with everything in life, it will turn around again.', \\\"We finally remember that our intentions weren't necessarily good and neither were your intentions. A discharge of our vows is a second step towards firbility.\\\", '[substeps] Write them down for future reference (or on a reference sheet). The more you understand the meaning of financial hardship, the greater your chances at survival.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}