{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"234f6e84-7ab2-5985-9b3b-6a2987bd6984","task_key":"default--validation--12878--variant-4b04a3519509","task_revision_id":"1","upstream_id":"12878--variant-4b04a3519509","short_description":"HellaSwag validation 12878","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to become a professional bachelor [title] End any serious relationship you may have with a female in a clean and respectable way. [step] As an expert bachelor, you will need to play the field. Therefore, marriage or any talk of marriage is out of the question.\",\"endings\":\"['As an advanced bachelor, you may not want a woman to see you strip, shave, file or cook. You may only want to mate women in a clean and respectable manner, or at minimum, make a conscious effort to do so.', '[substeps] In a relationship, the woman is considered a professional bachelor if the relationship is serious enough, both you and the woman are of sound mind and are comfortable with working together. [title] Stop doing things by the time you are ready to commit.', 'After you have a solid form of communication that has no feelings of detachment, this becomes clear from the outset. In general, refrain from any association that might be influencing your decision.', 'Follow these break-up guidelines : [substeps] Make break-ups fast and final. Do not attempt to hang on to the relationship, even in a friendship capacity, as this will only weigh on you emotionally and affect your ability to be a true professional bachelor.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}