{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"175ac220-9b18-50d1-9d22-5cd614cba22d","task_key":"default--validation--12522","task_revision_id":"1","upstream_id":"12522","short_description":"HellaSwag validation 12522","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to change a upvc door lock [title] Identify the kind of lock you have. [step] There are different variations for upvc door locks, so initially, try to identify the brand of the lock on your door. Some common brands include avocet, fulltex gu ferco, mila, roto, and yale, just to name a few.\",\"endings\":\"['[substeps] Otherwise, try having someone walk in and lock your door for you. See if the locksmith can help you; by hiring someone else to do it, they should be able to reset it just like you can.', '[title] Break the lock with a screwdriver. [step] After you have identified your specific brand of lock, breaking it by turning the screws in the opposite directions is the safest response.', '[title] Check the expiration date on the door. [step] Some brand names will expire, so call the supplier after you check that model of lock.', 'Knowing the brand of the lock will help tremendously in finding a replacement lock. [substeps] Upvc door locking points can come in various styles (hook, deadbolt, pins, etc.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}