{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5a9e2792-2e60-5cd2-8de7-3335923651e1","task_key":"default--validation--17627","task_revision_id":"1","upstream_id":"17627","short_description":"HellaSwag validation 17627","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to protect your pearls [title] Use a soft bag/pouch to store them. [step] Okay, so pearls can get scratched really easily. So, to store them use a soft bag or small but soft pouch.\",\"endings\":\"[\\\"[title] Take them out of the box immediately. [step] If they are broken or tear, take them out so they won't get in your way.\\\", 'These can be stones and shells, plant bark and various other objects. In the end it will be the right size and shape to keep your pearls in.', 'It is up to you how you use each bag/pouch. You want something that is comfortable and automatic to drop them into, so choose a small bag/pouch that has a low bottom so it can easily get back in after each use.', 'You can also use a clean soft cloth. What we are basically doing here is avoiding the pearls to get scratched.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}