I had a problem installing blend 3 on vmware player 2.5.2 build-156735. Most of the controls on the setup window did not appear. After some googling I found that the problem was on the graphics driver. It seams the driver has problems when combining 3D and 2D content in directx.
As suggested the workaround is disabling the 3d support. I did this using dxdiag tool that can be found on the system32 folder. Just type dxdiag (if you have dx9) on the run textbox and it should take you there.
You only need to disable it to run the setup. When can enable it back afterwards. If you find any problem when using it, specially when mixing 3D content just repeat the process.
Have fun,