{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"462b1222-0342-5b49-850a-12a0fbe074fc","task_key":"default--validation--37063","task_revision_id":"1","upstream_id":"37063","short_description":"HellaSwag validation 37063","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to store knee high boots [title] Zip up the sides of your boots in readiness for storing. [title] Roll up an unwanted magazine and insert into one boot. [step] ( you will need to hold the boot up as you insert the magazine.\",\"endings\":\"[') [title] Secure the magazine between the rear part of the boot and the sole of your foot using duct tape or dental floss, as if you were freezing. [title] Clamp down firmly onto the boot.', ') [substeps] Keep the magazine between the back and bottom of both your left and right boots. This will help to protect everything on both boots.', ') immediately it will uncurl and fill the boot shape. [title] Roll up a second unwanted magazine and insert it into the other boot.', ') [title] Place the magazine down on a flat surface to reduce the chance of the magazine or boots getting rolled up. [title] Pick up a very light object such as a book or newspaper.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}