{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c33f30e5-68d2-5e66-87fe-38aafce32a1b","task_key":"default--validation--12687","task_revision_id":"1","upstream_id":"12687","short_description":"HellaSwag validation 12687","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to leave home for the first time [title] Secure a place to live. [step] Perhaps the first step you should take when making preparations to leave home is to secure a place to live. Ultimately, without a place to live, you'll have a tough time managing a job, school, or anything else in your life.\",\"endings\":\"['As a result, make this search a priority. [substeps] Talk to friends or family in the area where you plan on living.', 'You can change the location of your residence by removing items from it or buying new items from someone who can find you there. [substeps] If you do not have a place to live, you may find others.', '[substeps] A friend or family member can help you pick an apartment. Have a list of possible accommodations for both you and your employer (if available).', \\\"Before you go anywhere, talk to your friends and family to see what's out there for you in your new housing situation. Some suggestions are : [substeps] The library, if you can afford it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}