{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"799c17f5-d333-52ba-85f1-e21354702039","task_key":"default--validation--15208","task_revision_id":"1","upstream_id":"15208","short_description":"HellaSwag validation 15208","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to take care of military boots [title] Use the on and off brush to remove muck from your boots, and to get the sides shiny. [step] First of all, get your on brush and put a bit of polish on it, although not too much. Don't wet the polish or anything yet.\",\"endings\":\"['Let it dry before proceeding with the next step. [title] Use the toe brush to insert up and down the inside of the boot, then (preferably on the longest side) up to and around the top edge, trying to get the cuff down smooth.', \\\"[title] Apply the polish on the boot, around the sides, heel and toe until the boot is dull [title] Take your off brush and brush really really hard for ages until the boot is shiny. [step] You won't be taking the polish off, but rather making it shiny.\\\", \\\"When getting a fresh pair of boots, put on a little bit, to start. [title] Once you've gotten the dirty boots to your liking, you might want to do a side brush on them (you'll never know it how dirty they usually get).\\\", 'Also, you will want to put newspaper underneath to dry before putting polish on the boots. After a good soaking, wipe off the excess with a microfiber cloth to get the coat looking shiny again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}