{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26b259d2-90af-57ea-a6ca-aa39430d0a42","task_key":"default--validation--46981","task_revision_id":"1","upstream_id":"46981","short_description":"HellaSwag validation 46981","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a perfume [title] Decide on a price point. [step] Perfume is a luxury item and its cost can vary hugely between products. Some perfumes are under 20 dollars while others cost hundreds or even thousands of dollars.\",\"endings\":\"['It is best to stick with one brand and limit your choices in a variety of different brands or styles. [substeps] The typical sized perfume pair is $5, but you can buy larger, less ornate ones as well.', 'Before you go shopping, decide how much money you want to spend on a perfume. [substeps] If the perfume is a gift, consider both your personal finances and your relationship with the recipient of the gift.', 'Before making a decision, make sure you are sure that your choice of an item would actually cost you hundreds or thousands of dollars. [substeps] You can measure the quality of the bottle or your fragrance.', 'Keep this in mind when deciding where to shop for great smelling perfumes. [substeps] Many perfume stores and online retailers offer personalized packages or deals for fragrances.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}