{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"514084e9-62cc-5c8a-a4ab-962d5017ba5b","task_key":"default--validation--28130","task_revision_id":"1","upstream_id":"28130","short_description":"HellaSwag validation 28130","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to put streaks in your hair at home [title] Pick out a hair color. [step] First, you will need to decide on a color for your streaks. If it's your first time, it's usually better to go with a color that is only slightly lighter or darker than your own hair color.\",\"endings\":\"[\\\"This will help to ensure that you get the right color. [substeps] When you're picking out a color, consider the face shape you want your streaks to have and the contrast to that.\\\", 'Regular color is easier to apply without using a lot of chemicals. For example, if you have dark colored hair, try keeping your hair dark at first and picking out a bold color shade to then experiment with with.', 'Many hair colors have a ph level of between 6.0 and 6.6, so colors like them tend to blend more easily. Otherwise, you may want to skip this section.', 'But, the color you desire is ultimately your choice. You may want blonde streaks, or pink or purple streaks.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}