Library Expects Symbol In Flat Namespace Although Compiled With Two-level Namespace

I load Python dynamically with dlopen and RTLD_LOCAL to avoid collisions with another library which by coincidence contains a few symbols with the same name. Executing my MVCE abov

Solution 1:

This question and your related RTLD_GLOBAL question both concern the semantics of the dynamic loader resolving undefined symbols in the shared libraries that it loads. I was hoping to find an explicit documentation reference that would explain what you're seeing, but I've not been able to do it. Nonetheless, I can make an observation that may explain what's happening.

If we run with verbosity, we can see that the python library is attempting to load two shared libraries before it fails:

bash-3.2$ PYTHONVERBOSE=1 ./main 2>&1 | grep -i dlopen
dlopen(".../python2.7/lib-dynload/_locale.so", 2);
dlopen(".../python2.7/lib-dynload/_ctypes.so", 2);

Given that the first one succeeds, we know that generally the dynamic loader is resolving the undefined symbols against the namespace of the calling library. And in fact, as you note in the comments of your other question, this even works when there are two versions of the python library, i.e. the dlopen()s done by the python libraries resolve against their respective namespaces. So far, this sounds like exactly what you want. But, why is _ctypes.so failing to load?

We know that _PyModule_GetDict is the symbol that was causing _locale.so to fail to load in your other question; and that it obviously works here. We also know that the symbol _PyBuffer_Type is failing here. What's the difference between these two symbols? Looking them up in the python library:

bash-3.2$ nm libpython2.7.dylib | grep _PyModule_GetDict
00000000000502c0 T _PyModule_GetDict
bash-3.2$ nm libpython2.7.dylib | grep _PyBuffer_Type
0000000000154f90 D _PyBuffer_Type

_PyModule_GetDict is a Text (code) symbol, whereas _PyBuffer_Type is a Data symbol.

Therefore, based on this empirical data, I suspect the dynamic loader will resolve undefined symbols against RTLD_LOCAL code symbols of the calling library, but not RTLD_LOCAL data symbols. Perhaps somebody can point to an explicit reference.