{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0be88ff6-bbec-58e3-ba0c-0560c311ff78","task_key":"default--validation--6679","task_revision_id":"1","upstream_id":"6679","short_description":"HellaSwag validation 6679","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to select and buy jewish wedding rings [title] Search the internet for information about jewish wedding rings and bands. [step] Don't be afraid to take your time and decide carefully. Examine your married friends' hands when you're with them.\",\"endings\":\"[\\\"If you start paying attention to people's jewish wedding ring designs, you will probably get a clearer idea of what you want. You might even ask them where they got their rings.\\\", \\\"What's they wearing? What bands do they wear? What are their marks or patterns? [substeps] Get a good idea of what bands they like and which ones they wear. Go around the ring with a notecard or piece of tape and write your own findings.\\\", \\\"Notice how quickly their hands move without work or planning. Are your hands overly warm? Are they too cold to handle? [title] Don't spend too much time with your wedding rings.\\\", 'You can create a custom design of images and drawings on your own. Some rings may be so tiny that you could have them printed on a sheet of paper and then hand ink them to the bride and groom if they ask you to.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}