{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"155b5f66-26ac-5038-b73b-d4b937f4298e","task_key":"default--validation--33964","task_revision_id":"1","upstream_id":"33964","short_description":"HellaSwag validation 33964","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to choose a minimalist engagement ring [title] Go for a smooth band for a complete minimalist look. [step] A smooth band with no embellishment, detail, or embedded stones is the most minimalist band you can choose. These bands generally have a setting of a solitaire, simple stone.\",\"endings\":\"['[title] Opt for an ice-blue or a sea-blue for a minimalist look. [step] These bands can be simple or minimalist.', \\\"The settings are determined by the carat, carat size, and carat diameter of the stone. They can be separated with the ring owner's initials, or the u.s.\\\", '[substeps] Smooth bands may have different finishes, such as polished, satin, brush, or hammered. Simple bands may be connected to elaborate and detailed stones.', \\\"Rounded bands with ornate embellishments and embroider numbers tend to be the most minimalist. [substeps] If you don't want a stationary, decorative ring, opt for a traditional touch.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}