{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1620dee4-b24b-5588-909c-30825c8b4d84","task_key":"default--validation--23497","task_revision_id":"1","upstream_id":"23497","short_description":"HellaSwag validation 23497","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to wear an engagement ring [title] Wear it on your left hand. [step] The ring is typically worn on the third finger of the left hand. This is the most common way to wear an engagement ring in many countries, especially in western cultures.\",\"endings\":\"[\\\"This dates back to the highly romantic idea that this finger has a vein that leads directly to the heart. That isn't true, but it's a nice thought for when you're looking down at that pretty ring of yours.\\\", 'If your choice is to wear an engagement ring on the fourth finger of the hand, however, it will vary from country to country. [substeps] The ring itself should be worn on the ring finger.', '[substeps] Rings come in a wide variety of shades and sizes, ranging from classic velvet to engagement rings made from precious metal. Novelty jewelry or designer engagement rings come in a variety of colors and patterns, and wear them on your right hand only.', \\\"[substeps] It's better to wear it on your right hand than on your left hand, because it's heavier. [title] Wear it with skinny jeans and boots to give some balance.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}