{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e92dfe7d-d95d-5d69-84e3-31978785646d","task_key":"default--validation--314","task_revision_id":"1","upstream_id":"314","short_description":"HellaSwag validation 314","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a lipstick holder [title] Cut the cardstock to the same size as the side of the box. [step] Take one piece of your cardstock and cut it so that it is the same size as one side of your paper box. Hold the cardstock against the edge to check the size.\",\"endings\":\"['[title] Make sure that the side is overlap. [step] This will be the side that the lipstick will hang down.', 'Then, cut the other pieces so that they are all the same size as this piece. [title] Make four cuts in each piece of cardstock.', 'Cut the cardstock so that it is a couple of inches longer than the length you want it to be. [substeps] The paper box should be at least three times bigger than the tulle box.', '[title] Glue down the cardstock onto the box. [step] Dip a glue stick or as many super glue strips as you need to go at the short end of the tube.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}