This post isn't really asking for a tutorial, but rather asking for some ideas. I've come up with a few ideas, but they seem cumbersome and unnecessary, so I was wondering if someone with more experience than me could help me out.

Also, my explanations suck, but I'll try to do my best.

So say I have an executable called X. At some point during its execution, X loads and uses (using dlopen and dlsym) a library called Y. In Y, there a bunch of functions that call the function called, for example, void *special_malloc (size_t). These functions come from another linked library (linked during compilation) called Z. Problem is, special_malloc contains some static variables within its scope, and when those variables are reset midway through a program, an invalid free occurs (special_malloc uses garbage collection). Now, what I want to happen is that whenever lib Z calls special_malloc, it instead calls the special_malloc defined in the executable X, so that these static variables are retained. Keep in mind that I have control of the Z library, meaning I can edit the source to fit my needs. Any idea?

Linux .so's are NOT Windows .dll's.

A Linux shared library doesn't hold "state".

A Linux executable loads a shared library ... and the EXECUTABLE holds the state.

If that's what you're getting at

I'm not sure I understand what you're getting at. Sorry, I'm not very good at programming, so if you could explain what you're saying it would be helpful.

Also I've never really used windows, so I'm not sure how DLLs work.