Is it possible to load DLLs and TX files from a different folder than where the main application is installed, e.g., a subfolder of my installation such as ODA\DLLs?

Yes, it is possible - to achieve this, their folders should be in the PATH variable. ODA DLL and TX modules work the same as any regular DLL.

For TX modules that are not linked (loaded dynamically with loadModule()), you can override OdRxSystemServices::findModule() and return the full path.