{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4b01ea2-83d9-5745-8ff9-dc79f2e47055","task_key":"default--validation--39398","task_revision_id":"1","upstream_id":"39398","short_description":"HellaSwag validation 39398","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to restore a boat [title] Make a plan. [step] Plan your work by breaking everything down into sections, like chapters in a book. Chapter one is planning.\",\"endings\":\"['This is where your business development can start. Analyze the navigation board, take all the necessary materials and then create your grand vision.', 'This step is not yet complete, so you need to prioritize moving toward part 1 of the project. This will help you tackle all your major projects faster, so that you can review your own work by taking each section and remembering to include the steps in that section over time.', 'The next phase might be stripping the boat in preparation for the work. [title] Prepare a work site, gather tools and supplies, carve out a dedicated time on a regular basis with your schedule, consult experts in the field, and also outsource certain components as needed to certain experts.', \\\"Plan your time on your own to avoid confusion and frustration with each chapter. Be sure that you aren't scrambling for ideas as you go forward so that you don't get overwhelmed by too many ideas.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}