{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d07adba5-bbae-5e04-bc56-da8903d86319","task_key":"default--validation--21247","task_revision_id":"1","upstream_id":"21247","short_description":"HellaSwag validation 21247","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to choose an unconventional engagement ring [title] Determine the colors they prefer. [step] Engagement stones come in a variety of different colors. Ask them, or take notice of their favorite colors and choose a ring that reflects it.\",\"endings\":\"['[substeps] The color wheel goes round and round, but the chosen color palette changes when you try new jewelry. Some colors that are unfamiliar to you may be too subdued for a traditional engagement ring.', 'For instance, a ring that is white or blue will always be white whereas a ring with a clever look or a unique theme will always be brown (gold) or red. [title] Try an unconventional ring : [step] A classic engagement ring will feature all the same colors.', 'Take notice of the current color of their jewelry and base your decision on that. Conventional rings are usually transparent, so consider getting a different color for your ring.', 'Accessorize with ties, buttons, necklaces, etc. [substeps] Look for a vintage or classic look when choosing an unconventional engagement ring.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}