{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff989edd-2ef1-5f05-9203-3ed19f2ab66b","task_key":"default--validation--26456","task_revision_id":"1","upstream_id":"26456","short_description":"HellaSwag validation 26456","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a dollar bill bow tie [title] Fold the bill in half. [step] To begin making this classic, somewhat chunky dollar bill bow tie, place an unfolded dollar bill on a flat surface. Then fold the bill in half lengthwise so that the two shortest sides meet up together.\",\"endings\":\"['This opens up the bottom of the bill. [substeps] Make sure that you use sewing pins and clips to securely secure the folds, otherwise you will end up with an uneven bow.', '[title] Fold the dollar bill in half widthwise. [step] After the first fold, make another fold, this time folding widthwise or \\\" hot dog style, \\\" so that the longest sides of the dollar bill meet up together.', 'Holding one strip of scotch tape roughly 1/8th of the length of the bill, fold both ends in half so that they meet up with the rest of the bill. [title] Cut the ribbon about ¼ \\\" away from the crease.', 'This will draw attention away from the bill and upward toward your eyes. Do this entire article in less than three minutes, and you should get three distinct ways to make the bow : [substeps] Fold the bill in half diagonally.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}