{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3f3621d8-e390-564e-8d14-878355346784","task_key":"default--validation--6642","task_revision_id":"1","upstream_id":"6642","short_description":"HellaSwag validation 6642","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress up bobby pins [title] Get some bobby pins, and slide them over the top of a sheet of paper. [step] Thicker paper, such as cardstock or an index card would work best for this, but you can use plain printer paper too. If you plan on painting several bobby pins the same color, consider separating them into small groups based on the color you will be painting them.\",\"endings\":\"['[title] With one hand, paint a shade of white over one of the bobby pins. [step] This will hold the whites of the bobby pins for a couple of seconds, while leaving the shades of pink out the window.', '[substeps] If you want to create a \\\" color blocked \\\" look, get some black bobby pins, and lay down strips of tape over them at various angles. [title] Paint the top of your bobby pins using a single stroke.', '[substeps] You should also keep the pins in different weights. For example, you could set the numbers 1 to 8 on a sheet of white cardstock, while the number 10 to 12 on a sheet of plain printer paper.', '[title] Wrap the bobby pins with tissue paper. [step] Turn one bobby pin up, so that the smaller end of the tissue paper sticks out near the top, and the thicker end sticks out to the left.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}