This blog post tries to capture some essential points to remember when using p/invoke from C#.
You can expose static methods of a C++ class using the __declspec(dllexport) modifier. The C++ compiler adds each method to the exports table of a DLL with a different name. Use the dumpbin utility (distributed with Visual Studio) to get the exported name of a method and add an EntryPoint field to the DllImport attribute. Dependency Walker is another good tool that shows the exports.
Read this article that describes the different calling conventions in great detail. If you haven’t messed around with calling conventions in your VC++ code, either by using __stdcall, __cdecl etc in your code, or by changing the compiler setting, you should remember to set the CallingConvention field in the DllImport Attribute to CallingConvention.Cdecl.
Callback using delegates
I refer you to this article that demonstrates passing delegates. In particular, it shows the right calling convention for .NET delegates i.e. __stdcall. If you have only the binaries for a DLL, you can tell C# to use CallingConvention.Cdecl using the UnmanagedFunctionPointerAttribute for the delegate. Note that you need to create a new instance of the delegate and pass that in your p/invoke call. Passing the method name can lead to a hard to decipher System.AccessViolationException.
Marshalling or type mapping
This article has a table of type mappings and other tips regarding marshalling. In case you need to pass or receive an array of bytes (char* in C or C++) or any opaque pointer, the IntPtr structure can be used. Use the Copy method from the System.Runtime.InteropServices.Marshal class to recover data as an array of bytes.
P/Invoke Interop Assistant
The P/Invoke Interop Assistant tool is very useful, it automatically generates C# p/invoke declarations for type and function declarations in C and C++. Red Gate has a free plugin for Visual Studio that can help you to find and contribute p/invoke signatures for well known APIs.