{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6facfc9c-fc5c-5073-96aa-f6cc48032f9d","task_key":"default--validation--26125","task_revision_id":"1","upstream_id":"26125","short_description":"HellaSwag validation 26125","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use hair removal cream on your face [title] Read the product label. [step] While the process might seem obvious, it's important to read the directions and make sure you understand them before using the cream. Different brands of hair removal cream have slightly different instructions.\",\"endings\":\"['[substeps] Additionally, this will allow you to learn of potential side effects and also check ingredients for anything you may be allergic to. Make sure the cream is designed for facial use.', \\\"[substeps] If you use the product twice a week or less than the amount instructed by the manufacturer, you may experience side effects. Talk to your doctor or pharmacist about this if you're concerned about using the product too often.\\\", \\\"The less information you can provide about treatments, the better prepared you'll be for others that may have similar instructions. [substeps] For example, if a hair care product specifically intended for your face requires your hair to be chemically treated, it may take months for the product to properly get into your pores.\\\", 'Regardless of what product you use, be sure to read the label carefully. There may be a specific ingredient that predisposes you to having an allergic reaction.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}