{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a563913d-83c8-5a6a-bbed-a57a73cfe3a8","task_key":"default--validation--1546","task_revision_id":"1","upstream_id":"1546","short_description":"HellaSwag validation 1546","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to strop a straight razor [title] Acquire a strop. [step] These are relatively simple to get. You can get cheaper models from general stores online.\",\"endings\":\"[\\\"These might cost you anything from $5 to $25. If you've been using a straight razor for a while, you'll want to invest in a strop that will last longer.\\\", 'They are designed to be effective for a range of purposes except shaving. They are ideal for when you want to really imagine shaving off the hair.', \\\"Here's what one could buy : [substeps] A strop is simply to simply hold one blade upright in one hand. Flat, sharp edges attach the blade to the body of the razor.\\\", 'Look for one that has crimp at the ends. Crimp is a soft, feminine material.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}