{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"523cf771-7b55-58d8-9e2d-9fa284596f27","task_key":"default--validation--2229","task_revision_id":"1","upstream_id":"2229","short_description":"HellaSwag validation 2229","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to use the omega paw litter box [title] Put the litter in the box. [step] Make sure it doesn't go above the grating. Use clumping litter.\",\"endings\":\"['[title] Make sure the clips hold the top and bottom closely together. [step] The clip from the top needs to go through the hole on the bottom, and then the clip must be pushed so that it locks into place.', \\\"The clumping litter makes it easier to unpack and level out and keeps the litter in a safer, more quiet location. It's also the most inexpensive, if available, litter box! [substeps] Never let older or neglected litter get in the way.\\\", \\\"Clumping is when you add cat litter to the bottom of the box, making for excellent, portable litter. [substeps] The best quality cat litter is one made with a starter material, but if you don't have a starter material handy, try a ground up litter.\\\", 'Put one tablespoon of cream cheese in the litter box and for each gallon of litter. Add two tablespoons of milk, one tablespoon of grass-fed cat food and one tablespoon of dijon mustard.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}