{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"181d772e-a47b-5d2f-9723-3ed07203ae22","task_key":"default--validation--47332","task_revision_id":"1","upstream_id":"47332","short_description":"HellaSwag validation 47332","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to temporarily fix a loose wire on your braces [title] Feel where the wire has come loose. [step] Wires can come loose over time, or due to chewy food. Make sure the bracket itself is still attached to your tooth.\",\"endings\":\"[\\\"If it's not, contact a dentist. [substeps] Look for loose nuts or fasteners on this bracket.\\\", 'Also, make sure that the wire has not come completely out of the bracket. [substeps] If the wire has come out of the bracket attempt to slide it back into place.', '[substeps] Insert a tiny fork between the bracket and tooth wall. This should provide you with an easy inspection of the wires.', 'If the bracket does not respond to your efforts to clean it out, pull it away and replace it. [title] Get a tooth brush and toothpaste.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}