{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cbf92a33-c20c-509d-8272-9e7725b750cb","task_key":"default--validation--2935","task_revision_id":"1","upstream_id":"2935","short_description":"HellaSwag validation 2935","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to buy a tanzanite online [title] Know the basics : [step] The four c's, as in diamonds, is applicable to tanzanites as well. These are: color, clarity, cut and carat. Unlike in diamonds, color in tanzanites is the most significant feature.\",\"endings\":\"['Use the image below to research how tanzanite works. [title] Most of the single ingredients in tanzanite will come in the glycolic acid diacetyl, an acid which is responsible for the lightening chemical called xalox calfenne.', \\\"[substeps] Carat: the cut is distinctive and measured by a' diamond' saw. Carat: the stone sits in a flawless tone (preventing cracking).\\\", 'The blue in a tanzanite is valued in terms of hue, tone and saturation. Clear, deep blue violet tanzanites are valued more than dusky ones.', '[title] Register online : [step] Just like with jewelry, you will be required to register for online exchange on the tanzanite website. You can register online in either amazon or itunes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}