{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"052a724a-2d84-5e5d-9089-ac28bd300d5f","task_key":"default--validation--40104","task_revision_id":"1","upstream_id":"40104","short_description":"HellaSwag validation 40104","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to host a good party [title] Figure out what type of party you want. [step] Having a guiding principle will make all the rest of your decisions simple and easy. Are you hosting a themed dinner party, or a blow-out with all of your friends? Do you want something small and intimate or the big banger that no one forgets.\",\"endings\":\"['[substeps] Maybe you want a temporary event rather than a full house party. Maybe you want to invite people over who already like your partyplan.', 'Are you inviting your friends to do something outside of your home? Consider a large home party or an outdoor party. [substeps] Keep in mind that you will need a great deal of space to accommodate all of your guests.', 'Ask yourself what kind of party you want to host before doing anything else--this will help you decide who to invite, what to buy, and what to prepare. [title] Invite your friends, making sure you only invite as many people as you can manage.', \\\"Everyone is susceptible to mood swings, so ahead of time, you'll want to feel the difference. [substeps] Be sure to assess your boundaries regarding how everyone should dress and behave, too.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}