{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c7d8558-a5bd-5933-a995-0617327cf242","task_key":"default--validation--39569","task_revision_id":"1","upstream_id":"39569","short_description":"HellaSwag validation 39569","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress like a lawyer [title] Think about where you live. [step] There are big differences in dress codes and expectations depending on where you live. Different regions of the united states have different expectations and dress codes.\",\"endings\":\"[\\\"If the person who hired you has expectations of your ability to dress professionally, you need to find ways to enact that expectation. [substeps] A candidate shouldn't wear inappropriate clothes.\\\", 'You will need to talk with an accountant or search online for some firsthand knowledge about the differences in culture and dress codes. [title] Make sure to wear skin care products and household products.', 'Before you go to work on day one, try to understand the norms where you are and follow them. If you are not sure, be cautious and dress formally and conservatively.', 'Pay special attention to which department you are being called to respect rules when it comes to changing into a suit. [substeps] Black and navy suits and white pleated shirts are the exceptions.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}