Registering phreplace DLL in Windows Vista
Warning: Please consider that this post is over 13 years old and the content may no longer be relevant.
phreplace is a VB6 DLL which is called from a vbscript plugin for PSPad. The script tests to see if the correct version of the DLL is registered on this computer, if not it registers it using regsvr32. This works fine in XP but not Vista where the script will error.
A while ago a user pointed out to me this article on Experts Exchange explaining why administrative rights are required to register a DLL in Windows Vista, in the hopes that we could automate the registration of phreplace in Vista.
Feeling defeated, I decided to explain why I haven’t been able to do this yet.
The fundamental problem is that Vista requires administrative privledges to write to the registry key required to register DLLs:
HKEY_CLASSES_ROOT\
However the user can register DLLs in their own section of the registry:
HKEY_CURRENT_USER\SOFTWARE\Classes\
So all is well right? Well not if you’re using VB6, you see when you run regsvr32, it calls the methods DllRegisterSever and DllUnregisterSever to add / remove entries from the registry, these methods are generated automatically by VB6 to register to HKEY_CLASSES_ROOT, and there is no way to change this.
Well, there is one way. Apparently the vbAdvance addon for VB6 allowed a user to customise DllRegisterServer / DllUnregisterServer, this sounds great, except that it doesn’t work. I tried modifying these methods according to the instruction, but the resulting DLL just wont register. I even tried registering the included example DLL in the download package, but this threw the same error.
I’m certain it must have worked at some stage, they wouldn’t have included an example that didn’t work, so I can’t figure out what’s different. Did it only work in Windows 98 or 2000?
So in summary, for as long as I’m using VB6 I don’t think I can change where phreplace tries to register the DLL, the only other workaround is to use RegOverridePredefKey to tell the registry to redirect any calls to HKEY_CLASSES_ROOT to HKEY_CURRENT_USER\SOFTWARE\Classes, this would work but I don’t think it can be done with vbscript, so I would need to add another executable to phreplace to use in registering / unregistering the DLL from the registry, seems pretty messy.
If you’ve got any tips for me (especially how I can customise DllRegisterServer) feel free to contact me.