{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fa1e5f67-9385-5c05-80d3-09185ecd42f9","task_key":"default--validation--23291","task_revision_id":"1","upstream_id":"23291","short_description":"HellaSwag validation 23291","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to weld lead [title] Prepare the lead for welding. [substeps] Before lead can be welded, it must be cleaned. This is especially important if you are replacing an existing piece of lead, such as an old pipe.\",\"endings\":\"[\\\"Be sure you don't use chemicals. [title] Remove the lead from the pipe each time you weld it.\\\", 'Clean the lead thoroughly with a wet cloth before welding. It is important that the lead is clean when you start doing welding.', 'Thoroughly rinse the lead with clean water prior to welding it. Prime and clean the pipe as you go.', 'Use a sharp knife to remove any debris or crusty lead from the area where the lead will be welded. In addition, you may want to consider using a piece of steel wool to completely polish the area.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}