Microsoft Windows uses a package called devkitPSP, which is GNU GCC based. This package houses everything needed for development on the PSP. It is a toolchain. The package includes libraries, headers, and tools for the PSP - some external libraries developed for the PSP are included.
DevkitPSP has a very nice installer which can be found here. This installer does not house any 3rd party software such as toolbars, malware, bulkware, etc. Click through to install devkitPSP, and optionally devkitPPC(wii,gc) and devkitARM(gba,nds). The root of all installations are in C:\devkitPro\*. However, subfolders are created in this directory to house installed items. The package houses examples, psp-gcc, and dependencies. One of the additional things the installer gives you is MSYS which houses make.
Some environment variables are set to be used with MSYS, however, using MSYS is not required. The addition for MSYS is to have a Unix-like environment which is favored by many developers. Additionally, using MSYS means the environment is set up already and environment variables do not need to be played with.
To not use MSYS, removed the related environment variables using System Properties. Makefiles use a constant variable, however. To set up everything to build correctly, set DEVKITPRO to `c:\devkitPro\msys\bin;c:\devkitPro;c:\devkitPro\devkitPSP\bin;` Some makefiles will have to be modified further to use del, copy/xcopy, rmdir, etc instead of rm, cp, etc. Most projects don't have that complex of a build, and issuing commands yourself using a single batch file will prove just as easy.