{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"85eb3500-0c52-5ef5-81fd-c713230cf028","task_key":"default--validation--38080","task_revision_id":"1","upstream_id":"38080","short_description":"HellaSwag validation 38080","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to rent a room in your house [title] Check your city's zoning bylaws. [step] Be aware that your city may have zoning laws about renting. These may prevent you from renting to unrelated persons without a permit or license, for example, or limit the number of unrelated persons to whom you rent.\",\"endings\":\"['[title] Find out if there are any local ordinances regarding rental apartments. [step] Do some research to see if particular municipalities regulate apartments or might allow rental as an option.', 'Do read further and follow all local and national laws. Do not rent to people with low-income status.', \\\"[substeps] When you're looking at a house, stay close to it, as you may find it boring and otherwise unpleasant to share space with other residents. Notices and notices generally must be given during the first few weeks after you move.\\\", 'Make sure that you will be in compliance. [substeps] For example, the city of north bay, ontario, requires landlords to secure rental licenses.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}