While it's not clear how much music would be available, it's reportedly willing to pay "some" labels per stream no matter how many ads run alongside those songs.
Amazon isn't commenting on the rumor. With that said, it's easy to see the company offering a free service. Although its existing pay-only services (Prime Music and Music Unlimited) have been relatively widely adopted with a total of 20 million subscribers as of 2018, it's unclear how many of those people actually listen. Prime Music is more of a bonus than something listeners seek out. Even if a free service didn't have a larger catalog than Prime Music, it'd have a much larger potential audience and would make Amazon more of a contender in addition to selling more Alexa-equipped devices.