We distinguish between "Functional" and "Content" output.
I think we really shouldn't have ceded this in the first place. An AI model isn't "output" it's a derivative work full of samples. It should follow the same rules that musicians have to. Only problem is there's been a concerted effort to defang IP for decades led by tech giants and heavily affecting the terminally online software folks who might be expected to care about GPL violations. In this respect the horses have already left and I'm not sure closing the barn door matters.
There's also the issue of enforceability. I could write a license that says 'if you want to use my library in your vibe coded slop you must first submerge your hand in boiling water for no less than ten seconds.' I don't think I'm going to be able to convince a judge that their failure to perform this entitles me to damages. Likewise I suspect that now that the practice is there a license like this isn't going to do anything to giants. The GPL came before anything worth stealing was licensed under it and helped shape the norms.
A more straightforward approach I've seen is a less ambiguous statement I see in books nowadays: this may not be used to train AI. I'd like to see versions of conventional licenses with this specific carveout.
I'll finally note that we have seen an attempt to do something like this before in the form of the AGPL which almost nobody ever uses. Contemplate why.