{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4dbecfaa-edfe-5553-b2cd-5ef446fe2b1f","task_key":"default--validation--7978","task_revision_id":"1","upstream_id":"7978","short_description":"HellaSwag validation 7978","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to prune holly shrubs [title] Master the basics of pruning. [step] It's not rocket science, but pruning is a bit more complicated than it looks. There are different techniques of shaping and specific places on the shrub where you should make the cuts, so don't just start chopping.\",\"endings\":\"[\\\"Your goal should be to take full advantage of the best section of the shrub so that you can't cut off too much. Therefore, you'll want to devote more attention to areas where you'd like to cut.\\\", 'Just focus on logging yourself in on the places where branches are divided into smaller sections and cutting the weaker-looking branches. [substeps] Avoid cutting down any of the existing branches throughout your shrub.', \\\"[substeps] Taking the branches on the back of the shrub to the ground will ensure a nice clean spread across the roots. If you're just to clean the shrub, pruning small areas is pretty simple.\\\", 'Before you get started, take a minute to review the fundamental terms and concepts pertaining to pruning. [substeps] Shearing is the act of maintenance where you simply trim the outermost twigs to control length.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}