{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bd58ca07-7794-5d1d-ac88-732f250f9a24","task_key":"default--validation--7445--variant-91a2fad06d16","task_revision_id":"1","upstream_id":"7445--variant-91a2fad06d16","short_description":"HellaSwag validation 7445","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to acid wash jeans [title] Select an old pair of jeans. [step] Acid washing jeans will remove a lot of the color. You should not acid wash a pair of jeans you are attached to.\",\"endings\":\"[\\\"Pick an old pair of jeans that you haven't worn since you've been acid washing jeans. Some old pairs of jeans are pre-washed so they will not react poorly to acid.\\\", 'Select an old pair of jeans for the acid washing process. [substeps] If you do not have an old pair of jeans, stop by a local thrift store.', 'Most unwanted pairs of jeans have been acid washed in pairs of six or nine different styles. Having an old pair of jeans that has been acid washed will require you to familiarize yourself with some of the different tricks of washing jeans.', 'In this situation, your jeans are worth a wash. [substeps] Selecting a pair of jeans with a few loose pleats will make them a cheaper alternative to jeans that have stains.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}