How to set up a MS-Win VM for development

From Qt Wiki
Revision as of 14:42, 30 March 2016 by EdwardWelbourne (talk | contribs) (→‎Get Productive: Measured my qt5 build size.)
Jump to navigation Jump to search

Many of us work on Linux so can't easily reproduce bugs that manifest on MS-Windows or test that features we develop work on that platform. The natural answer is to have a virtual machine for that use. I'll here talk through the steps to set up a virtual machine, running Windows 8.1, for building Qt and a simple test program with MSVC 2012 and 2015. I'm using VirtualBox (because I'm using Debian, which has easy packages for it) but I'm sure other virtualisation approaches shall be similar.

See also:

Basic installation and set-up

I assume you have MS install media for the version you want; put it in your machine's drive ready to boot from. Get your virtualisation system to give you a new machine ready to boot, with at least 45 GB of disk and plenty of RAM. Boot the new VM from your install media, go through with whatever makes sense there.

When it's time to create a user account on the machine, the installer may try to make you use a Microsoft account; if you don't want to use an existing account for your VM, or if you have no such account, you can bypass this by opting to "create an account" (which you won't actually do). The form that takes you to has a "Sign in without a Microsoft account" link at the end, which lets you just fill in user-name, password and hint to create a local account on the machine, not connected to anything else.

You may be invited to upgrade to a newer version of MS-Windows; don't be shy of declining, if what you actually want is the version you're installing. You may have to say "do it later" and then "decline the update" to get there, and it'll probably hassle you to upgrade for ever after, but it will let you say no.

Once you're done installing, remember to take out the install media - you don't want to boot into that next time your real machine reboots ! It's probably installed a bunch of apps for which you have no need. This is a developer machine, not the home entertainment platform it expects you to want it to be. The screen's bottom left corner will get you to the "start" system; you can right-click on any toy in there to get a menu that'll often include Uninstall. Some toys may lack that entry; you can at least unpin them from start and from the task-bar (at the bottom of the usual desktop screen); or you can change the size. A down-arrow at the bottom of the start screen shall get you to a full list of apps installed; go through that weeding out any junk you don't need. My estimate of how much disk space you need (above) is based on what I ended up with; I uninstalled everything I didn't need, at this point; so, if you don't, you may need slightly more than my estimate.

After install you probably need to do some reconfiguring. It'll have set

  • time-zone based on your choice of language and
  • screen resolution to some default;

If you have cause to change those, do so in the Control Panel (found via the start system, via screen bottom left corner, or via the setup app). Once you have that open, look for:

  • Clock, Language, and Region → Change date, time, or number formats
  • Appearance and Personalization → Adjust screen resolution

You can also futz with other stuff to taste. It may forget some of this on first re-boot, but I find repetition usually bludgeons it into submission.

Mounting a partition shared with your real machine

I find it useful to share some disk with my real machine, via which to move things to and from the virtual one. Your virtualisation system hopefully has some way to expose a local directory as a "network" share to the MS-VM. You can potentially do this several times, if you need to share more than one local folder.

To do this in VirtualBox, you need to get the "guest additions" ISO image; on Debian that comes as the virtualbox-guest-additions-iso package; or you can download it from Oracle. In the VM's container, Devices → Optical Drives → Chose a disk image and select that ISO image. Then Devices → Optical Drives should list the image; check its check-box. In the guest, the file-system Explorer should now expose a drive described as "VirtualBox Guest Additions"; double-click to run its installer. This tends to hide behind Explorer; dig it out and Next your way through it, accepting dialogs as they come up. That'll end with a reboot. In Devices → Optical Drives, you can now unselect the Guest Additions ISO.

You only have to do that once on any given VM. For each folder you would share, you can now configure it as a drive: Devices → Shared Folders → Shared Folder Settings … brings up a dialog. Right-click Machine Folders in the tree display; an Add Share dialog comes up. (Alternatively, to the right, there's a blue folder icon with a green + on it; it'll also Add Share.) In Folder Path, drop-down to select Other; a file-chooser comes up. Pick a directory you want to share with the VM. Back in Add Share, it'll fill in a folder name to go with that; edit to taste and select any check-boxes that seem apt; I usually just check Make Permanent. OK and you're back to the Shared Folders Settings dialog; OK and you're done.

Once you've configured a share, the MS local file-system Explorer should be able to see it; you need to configure it to mount this share as a local drive (or you can just access it as an UNC share). For VirtualBox, that shows up under (in the left panel) Network → VBOXSVR; open that to expose the share. Right-click on the share to select "Map network drive…" Select a drive letter you want to use for it, check suitable boxes and go. You should now see your shared directory as a drive in Explorer.

I usually share my directory of bare repositories (via which my working repositories mostly interact with upstream), which includes a sub-directory of noddy scripts for use to set PATH and other environment variables suitably for the Git Bash shell and for the Command Prompt I'll be using below. Speaking of which…

Configuring your Command Prompt

I use a Git Bash window in which I do all my git activity and everything Unix-ish; the default window comes up with a sane enough environment, but I could have a file to source for further hacks if I needed it. I have a separate Command Prompt window in which I do all the configuring, building and running of what I've built. This keeps Unix-ish complications to the environment separate from MS-ish ones.

For the Command Prompt, I have one toolchain-independent setup script and, per toolchain, a separate script to configure for that. We need to add suitable entries to PATH, LIB and INCLUDE. The first of these should have some sensible things in it already; in the raw Command Prompt (but not the Developer versions) the other two start out empty. Note that, while PATH deals well with quotes round entries, INCLUDE doesn't - even if an entry has spaces in it, don't put quotes round it. I believe LIB falls with INCLUDE in this; skip the quotes round its entries.

If you run into problems because a command is needed but not in your PATH, it can be constructive to use the Git Bash command-line's find command (being sure to search case-insensitively, with -iname rather than -name), e.g.

  • find /c/ -type f -iname ml64.exe -print 2>/dev/null

is how I found the missing entries in the VS2012 command prompt's PATH, that I needed to build OpenSSL. You can do similar to find missing headers or libraries you might need to add to INCLUDE or LIB.

Toolchain-independent set-up

You'll need the bin sub-directory of your perl installation in your PATH: this should have been arranged for you by the installer. Make sure the MinGW version of perl (installed with Git for Windows) isn't in PATH earlier than this; that version handles opening to a pipe poorly if the piped command has backslashes in it.

You'll need the lib sub-dirs of your Git's mingw (usually C:\Program Files\Git\mingw\) and your OpenSSL installation in LIB; you'll also need OpenSSL's include sub-dir in INCLUDE. (Remember: without quotes.) The bin sub-dirs of those should go late in your Command Prompt's PATH, so that you can run programs you have built, if they need the DLLs found via LIB.

If you've installed Jom, you'll want the directory containing jom.exe in your PATH; and you'll want to set NINJAFLAGS=-j4, tuning the 4 to be how much process-parallelism you want jom to exercise.

You'll need a qt5 checkout for its gnuwin32\bin sub-dir, which you need in your PATH. It probably makes sense for your PATH to include the bin sub-dir of wherever you build qtbase, to pick up qmake and related commands, if only for ease of use from the command-line if you need to run them manually.

Toolchain-dependent set-up

The "Developer Command Prompt for VS2015" or similar for VS2012 should in principle make a toolchain-dependent script redundant; however, I find their environments contain entries specific to Visual Studio (as distinct from MS Visual C++) that I don't need; and they lack the architecture-specific directories for at least one command, ml64, that the OpenSSL build needs (see the use of find, above). Inspecting the environment variables used in these Developer Command Prompt windows is, at least, one way to find useful directories to include in the PATH, LIB and INCLUDE set by such set-up scripts.

Some of what's needed is under the installation directory of the tool-chain; so I set that early in the script in a variable VSINSTALLDIR. Various other things are needed from the "Windows Kits" directory, typically a peer of VSINSTALLDIR (but sometimes one is under C:\Program Files\ while the other is under its (x86) variant); so I record this directory as KitDir.

You'll need PATH entries, under %VSINSTALLDIR%, for VC\bin\x86_amd64 (for 2012 on 64-bit) and VC\bin; note that the cross-compilation directory should come first, when needed. You'll want matching VC\include and VC\lib entries in INCLUDE and LIB, likewise. You'll need PATH entries under %KitDir% as well: for Windows 8.1, those are 8.1\bin and 8.1\bin\x86 (we suspect the bin\x64 you'll find there is an IA64-specific cross-compilation detail, like the bin\arm one, to be ignored unless you're actually doing that). You may also need INCLUDE and LIB entries from the kits, possibly even from under 10\Include and 10\Lib (even when building for 8.1); see what compilation and linking commands fail and use find from the Bash Shell to discover where files with the missing names are hiding; use common sense to chose among them, if there are several.

At the end of the tool-chain setup script, if not using the Developer Command Prompt, you need to run a %VSINSTALLDIR%\VC\vcvarsall.bat script. This sets up other weird stuff that MSVC needs; you *shall* get incomprehensible errors without it. MSVC 2012 deals with 64-bit as cross-compilation, for which you'll need to pass argument x86_amd64 to VC\vcvarsall.bat when you run it.

Installing the tools you need

We'll be installing MSVC 2012 express and 2015 community, Strawberry perl and OpenSSL. (In place of Strawberry perl, you could use ActivePerl; the latter's free license is limited to "non-commercial or non-production" use. If that's suitable for you, by all means use it instead.) For MSVC and perl, the links here should take you to a page with a suitable download link; or your favourite search engine should recognise the names just given, in case my links have gone stale. You might also want to install Jom; see its own wiki page for details. Each MSVC install takes a fair amount of time; be ready to find some way to amuse yourself while they're busy.

For MSVC 2012, you probably want wdexpress_full.exe, the self-installer. This doesn't offer you options, it just does what it does. If you're feeling adventurous, you might want to fetch the iso (possibly to your host instead of the VM; then expose to the VM as a mounted drive) and do a custom install.

For MSVC 2015, the installer offers you options; select a Custom install. It'll then ask what you want to install. Uncheck "Windows and Web Development". At the bottom of the list, open Common Tools and check Git for Windows. At the top, expand Programming Languages → Visual C++ and check MFC for C++ and Common Tools (for C++). (The git this provides comes with a version of perl that can't cope with open(P, "|cmd") if the cmd has any backslashes in it, which it typically does in what we'll need for OpenSSL below; which is why we need Strawberry perl.) If you don't install MSVC 2015, you'll need to separately find and install Git for Windows.

Once you've got Git for Windows installed, it's worth going back to Start and visiting the list of apps at your disposal; find the Git Bash app and right-click to pin it to the taskbar; you might also want to do similar for the developer versions of the Command Prompt, for MSVC 2015 and for MSVC 2012. This is also a good time to configure git on your VM; tell it to use autocrlf and let it know your name and e-mail address, at least.

The web page for Strawberry perl has an obvious box labelled "Recommended version" with a link for each of 32-bit and 64-bit; click the one appropriate to your VM. Once the installer comes up, do what it needs. Its install path can't have spaces in it; so don't try to put it under C:\Program Files\ ! Its default install location is C:\Strawberry\, which works fine for me.

If you're going to be building QtQml, you'll also need python installed: see the link for a suitable installer or rummage on python.org if it's moved. Python 2.7 fitted within 70 MB of disk space.

Build and Install OpenSSL

You've actually got two versions of OpenSSL already:

  • Git for Windows includes a MinGW sub-system that includes OpenSSL libraries and might even include headers; but I found mine incompatible with my system headers;
  • Strawberry perl also includes OpenSSL libraries, possibly with similar problems; and without headers, so you can't build against it anyway.

(You need to avoid having those versions show up in your LIB or INCLUDE variables. This should be easy.) There are also places you can install ready-built binaries from the web; but we'll be building it from source. Who do you trust ?

Visit the openssl.org source download page. You can either follow their github link to get a clone URL or download a tar-ball. Pick a place you'd be happy to build OpenSSL and either clone the git repo or unpack the tar-ball. If using a git clone,

  • be sure to cd into it first (doing git config in the wrong place has unintended effects);
  • use the (repo-local) git config settings the project recommends to handle line-endings;
  • ask git branch what branches are available; a recent OpenSSL_M_m_p-stable (with digits for M, m, p indicating Major, minor and patch level releases) branch is probably what you want; or ask git tag and checkout the most recent official release, OpenSSL_M_m_pR for some letter R.

Your instructions are in INSTALL.W64 or INSTALL.W32; the former refers to the latter for the many things that are done the same for both, but what you need for 64-bit is all in the former anyway. Since nmake install won't work (the generated makefiles assume way too much Unix-ish-ness), you'll end up doing an install by hand (I'll come to that shortly); but it's still worth passing Configure a --prefix=… saying where you do want to install; I chose --prefix="C:\Program Files\OpenSSL\" for this.

Once you've built and tested, it's time to install. Create your chosen prefix directory and, under it, subdirectories bin, lib, include and include\openssl. From under your build directory, copy (or move) to under your prefix as follows:

  • inc32\openssl\* into include\openssl\
  • out32dll\ssleay32.lib and out32dll\libeay32.lib into lib\
  • out32dll\openssl.exe, out32dll\ssleay32.dll and out32dll\libeay32.dll into bin\

If you have appropriate privileges, you can do this from your Git Bash shell; otherwise, you can get past the privilege problems (e.g. if your prefix is under C:\Program Files\) by using the file-system Explorer; it'll prompt you for confirmation at each step, but it'll let you do the job.

Once you've done that, assuming your prefix isn't inside your build directory, you can blow away the build directory; you don't need it any more and it'll free up some disk-space.

Compare and contrast: Compiling OpenSSL with MinGW

Get Productive

At this point - with MSVC 2012 and 2015, Strawberry perl, Jom and OpenSSL installed - I've used 26 GB of disk. That's up to 27 GB by the time I've checked out qt5 (just the super-module, for the gnuwin32\bin sub-directory I'll be needing in my PATH), qtbase (to build it) and my noddy test application (that exhibits the MS-Win-specific bug I'm investigating). You'll need 3 or 4 GB more if you check out all sub-modules of qt5. This is more than the 25 GB MS's installer recommended I have; but I let myself have 69 GB just to be on the safe side. This is a good moment to shut down your VM and take a snap-shot, before you manage to break it.

You should now be ready to build Qt. Run configure.bat (ideally from a shadow directory rather than from within the source tree) with -platform win32-msvc2012 or win32-msvc2015, as appropriate (if on 64-bit, don't worry about the win32 misnomer), along with whatever other options take your fancy, as usual. (That, just for qtbase, eats another 3 GB of disk.) Then run nmake and go take a break while it does all the hard work (and uses up another 10 GB of disk; for a full build of qt5, keep 25 GB of disk in reserve).

Retrieved from "https://wiki.qt.io/index.php?title=How_to_set_up_a_MS-Win_VM_for_development&oldid=23614"