{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"24c4c7e4-2399-55cc-b729-41e5fb2d2e5a","task_key":"default--validation--182","task_revision_id":"2","upstream_id":"182","short_description":"HellaSwag validation 182","config":"default","split":"validation","body":"{\"activity_label\":\"Sharpening knives\",\"ctx\":\"Then he takes a small stone from the flowing river and smashes it on another stone. He starts to crush the small stone to smaller pieces. he\",\"endings\":\"['cuts the center stone in half and blow it on to make it bigger.', 'grind it hard to make the pieces smaller.', 'eventually brings it back into view and adds it to the smaller ones to make a small triangular shaped piece.', 'starts to party with them and throw the pieces by hand while they celebrate.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}