Step-by-step instructions on how to compile your own ICU under Windows:
Compiling-ICU-with-MinGW Compiling-ICU-with-MSVC