{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"976f53e4-4318-5d97-8cef-603606e22e70","task_key":"default--validation--43546","task_revision_id":"1","upstream_id":"43546","short_description":"HellaSwag validation 43546","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to buy a treadmill for your home [title] Keep in mind your financial limitations. [title] Think about which use you will give to the treadmill. [step] For example, if you use the treadmill just for walking, a basic model would be suitable for your needs.\",\"endings\":\"[\\\"If you use the treadmill a lot, you should go for a high durability treadmill. [title] Learn a treadmill's controller.\\\", 'A more complex model is better for activities that require only walking. The less complex the model, the more gear you will need to share with your neighbors.', 'If you think you may someday only use it for sports, a summer treadmill is a great choice. [title] Get the right type of pedal length.', 'They can help you with push-ups, squats and more should be fine too. [substeps] Decide what type of model you will be using as well as how much you will charge.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}