{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2dd14da1-b312-5cf6-a13e-8524b0de181c","task_key":"default--validation--6403","task_revision_id":"1","upstream_id":"6403","short_description":"HellaSwag validation 6403","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply eyeliner (men ) [title] Use an eyeliner pencil. [step] The most common method for applying eyeliner is by using an eyeliner pencil. These pencils have soft, malleable points that draw on smoothly and have a rich, dark, creamy color.\",\"endings\":\"[\\\"You don't need an eyeliner pencil unless your application involves liquid eyeliner. Most eyeliner pencils have a clickable brush, without a stipler attachment, to create eyeliner.\\\", 'They are easy to apply without applying too much make up, which is flattering for both men and women. [substeps] There are eyeshadow shades available that will appear far more flattering with eyeliner, especially a dark brown.', 'Extend the pencil in a straight line across your eyelid over your nose, on both sides. [substeps] Be sure to hold the eyeliner line at a much more dramatic angle, because this allows you to see if the line is longer in one area then short (orange paper) and more dramatic on other portions of your face.', 'Eyeliner pencils are easy to come by and can be purchased in most drug stores or supermarkets in the personal hygiene section. They come in multiple colors and shades, but black is standard and is the most masculine.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}