{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6246608-c66b-506e-a1c7-8726d2513d00","task_key":"default--validation--31348","task_revision_id":"1","upstream_id":"31348","short_description":"HellaSwag validation 31348","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to go through your jewelry [title] Throw or give away all your broken jewelry. [step] If you have necklaces that are broken or earrings that are bent out of shape, throw them away. If you really love the jewelry and wear it often, you could try to get it fixed by your jeweler, but it will cost you.\",\"endings\":\"['[title] Have your jewelry analyzed before you buy it. [step] You should have assessing tests performed and certified by your jeweler.', \\\"It depends on your will whether you want to fix it or replace it. [title] Take out any jewelry that you've outgrown.\\\", \\\"It is better to go through a designer's store or a few chain stores, than to pawn your jewelry. [title] Go shopping for items you can't find anymore.\\\", 'A good bet is to go online and, if possible, check for blank pieces to throw away. Obviously, there are some on the net that are worth more than 99 cents a piece.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}