{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87f4f3d7-02ae-5713-9f8b-652b8c606276","task_key":"default--validation--11924","task_revision_id":"1","upstream_id":"11924","short_description":"HellaSwag validation 11924","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to choose database software [title] Determine what you want the database to do. [step] Base your decision on how the software meets the needs of your organization, rather than familiarity with the platform that the program is developed in. [title] Make sure the database is large enough with room for growth.\",\"endings\":\"['[step] Most likely, multiple users will need access to the database at the same time, so the database should support multiple users. Even if you start with a single user, plan for growth of your organization, and expect others to need access to data.', '[step] Online options are more competitive, so it is best to buy a database for a wide range of purposes. Every database is different, and you may need to tell your company where to get a database where you need to be strategic.', '[step] If it is too small, your database will become outdated and unable to generate searches. [title] Consider reducing the number of search engines.', '[step] Some of the variables set up in the database create value, so make sure your database includes enough data to generate growth in your organization. Therefore, it is very important to have a database that is huge enough for your employees to share and have data built in, allowing for improvement.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}