{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7152686b-3704-5fde-b4b0-e2529d2d2202","task_key":"default--validation--34066","task_revision_id":"1","upstream_id":"34066","short_description":"HellaSwag validation 34066","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to bid on government contracts [title] Search the federal business opportunities website (fbo). [step] Go to. Federal agencies use the site to list their contracting opportunities.\",\"endings\":\"['If you have an internet browser but have an internet connection enabled then you can perform an internet search for \\\" government contracts. \\\" [title] Search the federal trade commission (fbo).', 'All federal solicitations worth at least $25 , 000 are published on this website. [substeps] Be careful of private websites that charge money to access their listings.', 'If you are in business with another business, you can use the search engine to find specific business opportunities and book a bid. [title] Type the location of your bid into the search box at the top left corner of the web page.', 'You can also find the latest information by clicking on. [substeps] The fbo app is available for windows xp.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}