{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"38fefded-2863-5c63-af71-2d33e8a0ba4b","task_key":"default--validation--46920","task_revision_id":"1","upstream_id":"46920","short_description":"HellaSwag validation 46920","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make loom band patterns without the loom [title] Choose your colors. [step] The single chain pattern makes it easy to incorporate as many colors as you want. Decide whether you want the whole bracelet to be the same color or if you want to make a pattern with more than one.\",\"endings\":\"['Cross the image in back so the main details are shown. [substeps] Make sure that the pattern you choose matches the colors on the loom.', 'You could alternate colors or incorporate every color of the rainbow. [substeps] You might want to count your rainbow bands to make sure you have enough of the colors you want.', \\\"This rule won't work for all bracelets, so choose as many colors as you want to use it. [substeps] Colors on the loom color wheel help facilitate the development of amazing patterns and holding up as many colors as possible becomes easier! The bands in this project show up to display on the necklace.\\\", 'Incorporate bands of the same color together with colors that are consistent. [substeps] Shades of yellow, orange, and gold work best.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}