{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"73da95a2-fe84-591a-9daa-1da94f0b70dd","task_key":"default--validation--37526","task_revision_id":"1","upstream_id":"37526","short_description":"HellaSwag validation 37526","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to create a history club [title] Decide on your club's focus. [step] Since you've decided that you want to start a history club, you will need to decide if you want your club to focus on a certain time period or region. You could also choose to study a particular topic, concept, theme, or a specific event.\",\"endings\":\"['Figure out what combination of genres you want to focus on and decide if your club publishes particular events. [substeps] You may decide that all your club music will have an enthusiasm for historic era events with generations as the focus, formations and incorporators as the lighting crew.', \\\"[substeps] Think about your club's emphasis on a particular subject or aspect of history. How can you understand the point of this topic and apply it without causing repercussions? [title] Provide a calendar.\\\", '[substeps] For example, you may want to study the concept of revolution, which would include events like the french revolution, american revolution, haitian revolution, egyptian revolution, etc. Alternatively, you could choose to study a specific event and focus on just the french revolution.', \\\"This will allow you to learn from the experience and a book idea. [substeps] A history club's focus will determine who you can match your club to in your days and days.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}