{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ffb97071-215f-5711-9805-c8fd4789bf00","task_key":"default--validation--47744","task_revision_id":"1","upstream_id":"47744","short_description":"HellaSwag validation 47744","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to organize a church bazaar [title] Schedule a meeting with the church officials such as the pastor, youth leader, church secretary. [substeps] Decide on what should be sold at the bazaar and who should help. Once these details have been figured out, then you can schedule the bazaar at any time you want.\",\"endings\":\"['[title] Go through the bazaar once a month. [step] Decide whether each should be individually bought at a moderate price.', \\\"This can be a weekday, a sunday, or a tuesday. Don't be hasty when scheduling a sale but if you plan to organize the bazaar for a special occasion, show it off to local businesses, church etc.\\\", \\\"Usually, the best season is during summer and it should not conflict with anything else that is important to your church family. [title] Add the event to the church's calendar and notify everyone about it.\\\", 'Once you decide on a time, schedule a meeting with the secretary. [title] Ensure the bazaar will be well-planned.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}