{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d06ffdc-38cb-5a68-a137-5219f765f7f4","task_key":"default--validation--43074","task_revision_id":"1","upstream_id":"43074","short_description":"HellaSwag validation 43074","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to create unique lyrics for a song [title] Avoid overused phrases. [step] There are many different phrases that are used frequently in song lyrics. Using these phrases may seem harmless, but some are so overdone that they may make your lyrics sound corny or meaningless to others.\",\"endings\":\"['Listening to several songs and trying different phrases is not healthy. It also risks scaring away others and possibly making you sound stupid.', '[substeps] For example, if you were trying to write a song about a band, a particularly silly phrase could be \\\" the band--and i couldn\\\\'t give more than that. \\\" it\\\\'s okay to use the phrase \\\" truth is...\\\", but try not to assume anything more from the context.', \\\"To keep your lyrics fresh and original, reflect on each line as you write it and ask yourself if you've heard the phrase before. If you aren't sure, look the phrase up online to see if it comes up frequently.\\\", 'If you are trying to produce a song without an audience, be sure that you save money and use proper grammar and punctuation when you use the phrase. [substeps] Avoid lyrics that seem overly detailed to the listener.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}