... like a patched system without a proper patch
Record-keeping == wrapping the changes into packages and using "special" build numbers for those packages. E.g. my own packages are:
root@riposo:~/works# ls /var/log/packages/*-[0-9]*C
of which only libxcb-util is such an old libraries package -- the other ones are contributions.
I would say that this is easily manageable.
Let us end this discussion here, shall we?
Kind regards, Dick