{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"78654f96-7551-534c-94ae-7352418bb124","task_key":"default--validation--3418","task_revision_id":"1","upstream_id":"3418","short_description":"HellaSwag validation 3418","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to draw in iphone notes [title] Upgrade your notes app. [step] In order to add drawings to notes in your notes app, you'll need to be running ios 9 or later. You'll also need to be using the upgraded version of the notes service.\",\"endings\":\"['[substeps] To turn on ios 9, click the email link for your iphone notes app. Right-click the email and select upgrade notes on current.', \\\"Each iphone app has different features that you'll need to upgrade. [substeps] Iphone notes' library interface will vary by app.\\\", 'You may be prompted to upgrade when you launch notes for the first time after upgrading to ios 9. If you\\\\'re not, tap the \\\" [\\\" button to view the folder screen, then tap the \\\" upgrade \\\" button in the corner.', '[title] Connect your iphone to your computer using usb cable and spotify. [title] Look for a drawing icon in your app drawer.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}