{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f69da0d-6d6e-5b33-b137-8e2e4c39b1ce","task_key":"default--validation--2522--variant-dd810048c1f5","task_revision_id":"1","upstream_id":"2522--variant-dd810048c1f5","short_description":"HellaSwag validation 2522","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to fix a scratched dvd [title] Assess the damage. [step] Decide whether your dvd can be fixed, or whether it has irreparable damage. [substeps] As mentioned above, disks with scratches that go deep will not be able to be mended.\",\"endings\":\"[\\\"If your disc's damaged protective coating won't come apart with simple copying, repair it. If your disc has an extremely stubborn coating on, it's likely that it will probably need surgery to repair in a timely fashion.\\\", 'To assess the damage, hold the disk up in the sunlight. If sunlight shines through the scratch, the dvd is unlikely to be mended.', 'If you see a particularly damaged disk, try cleaning the disc as soon as possible with a solution that contains baileys acetone. [title] Consider repairing the disc before bringing it home from the store.', 'Even had the disc be repaired, a scratch can likely cause damage to the dvd. Repair your disc with constructive force by scratching scratches or by fixing the damage that will not be repaired.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}