{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4364e2a9-e7f2-5d09-9a88-0d3c1275f601","task_key":"default--validation--45409","task_revision_id":"1","upstream_id":"45409","short_description":"HellaSwag validation 45409","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to become a wedding photographer [title] Purchase quality equipment. [step] Wedding photos are heirlooms the family will treasure for years. To produce high quality work, you should use high quality equipment.\",\"endings\":\"['Start with mid-range products--you can always upgrade later. Talk to retailers and other photographers to see what they recommend.', 'You will need to take countless photos during the wedding and then bring them home. [substeps] Set aside enough time throughout the year to stock up on quality equipment.', 'Good equipment includes an array of different methods of recording digital photographs. [substeps] Check over your equipment list and think about booking a caesarean license when buying equipment for yourself.', 'Your time and energy are probably plentiful in your office. Here are a few ideas to get you started : [substeps] A picture frame paper printer a digital camera phone book foam board umbrellas [title] Work hard.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}