{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a532b59-2c6e-5bee-9d31-4b38499c470a","task_key":"default--validation--6498","task_revision_id":"1","upstream_id":"6498","short_description":"HellaSwag validation 6498","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to create a massage room in your home [title] Consult city ordinances before creating a home massage business. [step] Some cities pose restrictions on the room and location of the massage room. Apply for liability insurance, if necessary.\",\"endings\":\"[\\\"If you don't have a license to practice in a city, check state or regional regulations. [title] Apply for a home massage company membership or business membership.\\\", '[title] Choose a room in your home that is free of activity and noise. [step] It is a good idea to choose a room without a window, if possible, because it is harder to control light and noise.', '[title] Choose the right room to practice in. [step] Choose the room where you will put your massage studio and office.', '[title] Establish useful demographics and defining areas of your home to practice. [step] Do you favor older or teens facilities? Is it appropriate to have different equipment? For example, funding needs will vary from city to city, so try to find options that fit with these needs.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}