{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c5f1cee7-fc9b-5c84-a828-63a6d2049ca9","task_key":"default--validation--35975","task_revision_id":"1","upstream_id":"35975","short_description":"HellaSwag validation 35975","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to install a new processor [title] Find your motherboard's documentation. [step] The number one factor that determines what processor you can install is the socket type of your motherboard. Amd and intel use different sockets, and both manufacturers use multiple socket types depending on the processor.\",\"endings\":\"['Make sure your motherboard has socket type included before you start installing your new processor. [title] Visit the amd site at https: // www.amd/download-files-for-new-computer-site.', \\\"You should use either socket type or socket size indicated in the information on the packaging. [title] Go online to extract your motherboard's documentation.\\\", 'The process is much simpler with amd. [substeps] The socket type will determine what processor to install.', \\\"Your motherboard's documentation will provide you the necessary socket information. [substeps] You cannot install an intel cpu in an amd motherboard, or vice versa.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}