{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c69fed7-166e-5b85-ba66-3c9b29898293","task_key":"default--validation--25190","task_revision_id":"1","upstream_id":"25190","short_description":"HellaSwag validation 25190","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to upgrade from an nvidia geforce graphics card in an asus laptop [title] Remember to save all of your work and turn off the device. [step] Once you've turned it completely off, make sure to unplug the laptop from the wall, if it is connected. If possible, make sure to also take out the battery from the laptop, just to be safe.\",\"endings\":\"[\\\"[title] Expect to see some great character icons and bright colors when you turn it off. [step] You won't see them for a while, although some will change features every few years or more depending on the platform.\\\", '[title] Disconnect the positive and negative wires. [step] Connect the positive and negative wires facing each other.', \\\"You don't want to be messing around with expensive electronics only to have your laptop start up and short circuit. [title] Select a screwdriver that best matches the screws on your device.\\\", '[title] Install the software wizard that came with your computer. [step] This will help you get the latest and most powerful software for your nvidia laptop.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}