{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa2c5fde-c631-5f44-b548-29e8364e83b1","task_key":"default--validation--47060","task_revision_id":"1","upstream_id":"47060","short_description":"HellaSwag validation 47060","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean colored leather [title] Vacuum or shake off loose debris. [step] Use a furniture attachment to remove all extra dirt and debris from your furniture. Be very gentle when vacuuming so you don't scratch the leather.\",\"endings\":\"['Also, be mindful of the decal. If the decal gets damaged, then your leather may become burnt or discolored.', \\\"It's more safe to vacuum cushions, fan covers, or coffee tables than to completely clean your leather. [title] Choose a vacuum cleaner that will prevent dust from getting into your upholstery.\\\", \\\"Shake dirt into a bucket or wide container. Shake it until you've collected all the dust and debris.\\\", 'Brush dust and debris off of other leather items, like purses and gloves. [title] Make a basic cleaning solution with mild soap and water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}