{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6bb96bcc-e14e-5603-a1dc-2e77705d996b","task_key":"default--validation--17289","task_revision_id":"1","upstream_id":"17289","short_description":"HellaSwag validation 17289","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to know if your razor needs stropping [title] Look to see what type of razor you have. [step] Only straight razors require stropping. However, the amount of stropping depends on the type of blade, the strop itself, and the condition of the blade.\",\"endings\":\"['Most razors people use today are not straight razors and do not require stropping. [substeps] You may choose to strop disposable razor blades.', \\\"[substeps] You can pass any kind of blade between 2 to 5 mm. Unless you're sharpening many blades, don't use a razor because the stropping will remove it.\\\", \\\"So if you have a razor that has been previously treated for stropping, you should see if you have any special blade coating. [title] Ask what the razor's age refers to, and if it has any special protective qualities.\\\", 'While the stropping has gps units inserted into it, the stropping transmitter is only used when the blade is switched between 24 to 72 hours for proper sharpening. [ citation needed ] [title] Set your blade to stropping mode.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}