{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"64e0ae91-4276-515e-b33c-8cedb3c79f2b","task_key":"default--validation--36484","task_revision_id":"1","upstream_id":"36484","short_description":"HellaSwag validation 36484","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to read pedigrees [title] Figure out the orientation of the pedigree. [step] Pedigrees can be laid out in a variety of orientations. Assess whether the chart is oriented either vertically or horizontally on the page.\",\"endings\":\"['Look at the orientation of the words to understand which direction it should be held. [title] Trace lines in the pedigree.', '[substeps] Pedigrees are generally placed at different heights, from 1 degree to 12 degrees. If the chart is tilted upwards on one height, then it tells you how high it is.', 'As your child grows and begins acquiring pedigrees, the chart you will need will be oriented horizontally and vertically. [title] Learn what each pedigree means.', 'You can also ask your book store employee to help you look for the chart. [substeps] Each pedigree may have a specific orientation, often with many different aspects, that the reader is most familiar with.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}