{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"56025cac-b96c-5539-8ba6-1fab49ec791a","task_key":"default--validation--42911","task_revision_id":"1","upstream_id":"42911","short_description":"HellaSwag validation 42911","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to paint nails like a pro in minutes [title] Pick the right polish. [step] Not all polishes are created equal. Some have potentially harmful chemicals, and others just look downright bad.\",\"endings\":\"['These details are crucial for your nails to have a winning color. The option you select depends on your nail polish hue, color, and particular nail variations (white is best, as is brown/blue of course).', 'There are a number of things to keep in mind. [substeps] Use a b3f, or \\\" 3 free \\\" nail polish and base/top coat.', 'Choose between : [substeps] Antibiotic with a possible bleaching effect and baking soda and baby powder mix. They contain glycerin and other chemicals used to remove dead skin pores and tear marks as well as minerals and skin gels (onion nudes).', 'Be careful about how much nail polish you apply to the nails, as nail polish remover is a very dangerous product. Nail polish remover will reduce the amount of the chemicals that are produced in nail polish paint, resulting in a cleaner paint job.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}