{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0edc4b92-ec4e-5b8e-9b1a-926869fa3ebe","task_key":"default--validation--17736","task_revision_id":"1","upstream_id":"17736","short_description":"HellaSwag validation 17736","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be street smart [title] Avoid wearing headphones while you are out. [step] While it may be enjoyable to listen to music while you walk around, it is best to stay as alert as possible. Not being able to hear your surroundings will make you vulnerable to accidents or attacks.\",\"endings\":\"[\\\"If you absolutely have to listen to music or other audio while you're out, use only one earphone or play it at a very low volume. [title] Do not look at your phone while you are walking.\\\", \\\"It's also too risky to be distracted while you are trying to focus on a particular task. Try to avoid leaving your headphones on until you have no choice but to move around a bit to get the most out of your listening.\\\", 'Always work at it: make sure to put noise pollution out of the equation, and then put it at a lower volume (like going up the ac, heating up a window, using an air conditioning unit). [substeps] If possible, wear headphones when you are out, and press down on the volume button.', 'Putting headphones on makes it even more difficult to slip in and out of your location. [substeps] You might think it a subtle way to keep yourself out, but if you are in a public place or in situations where you can slip in and out unnoticed, you can still avoid mishap risks by wearing headphones.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}