RunningBuildbotOnWindows – Buildbot

RunningBuildbotOnWindows – Buildbot

Buildbot on Windows - Installation Instructions.

Buildbot runs on Windows, as both a slave and a master. The master can run on any convenient platform.

Prerequisites for a slave and for a master

• Python 2.x. Use the latest 2.x version of Python.
• PyWin32. Match the version with the python you installed.
• Twisted. Match the version with the python you installed.
• Zope.Inteface. This will be installed using setuptools and then easy_installing it via "C:\python27\scripts\easy_install zope.interface".
• Buildbot, of course. You need 0.8.2 or later.

Additional Prerequisites for a master

• Jinja2. This will be installed using setuptools and then easy_installing it via "C:\python27\scripts\easy_install Jinja2".
• PyOpenSSL. This will be installed using setuptools and then easy_installing it via "C:\python27\scripts\easy_install PyOpenSSL".

For the remainder of this document we will assume python was installed in "C:\Python27". Please adjust accordingly if you install to a different path.

Install python, pywin32, and Twisted for all users. On Windows Server 2003 it was found that this MUST be done while logged in as the actual "Administrator" account, NOT as a regular user with Administrator privileges, because of the way filesystem permissions are inherited during egg installation. The buildbot service itself need not be, and probably should not be, a privileged account (see below). When running python setup.py, be sure to use the full path to the version of python you expect to run buildbot (It is ok to have multiple versions of python installed). The pywin32 and Twisted installers will be matched for the version of python that will be running buildbot.

Install Buildbot itself with

C:\python27\python setup.py install 

The following steps were successfully used on Windows Server 2003 SP2 (32 bit) using python 2.7 for buildbot 0.8.3p1. They should work for later versions of buildbot. Note the use of an absolute path to the python executable during installation; this is required if you have multiple side-by-side installations of python

1. log in as Administrator directly (ctrl-alt-delete from the login screen) Doing this as any other user, even users with Administrator privileges, will cause the eggs to be installed with incorrect permissions and a non-privileged buildbot service will fail to launch.
2. download the above packages via firefox, for example into c:\Documents and Settings\Administrator\Desktop. IE will try to turn the egg file into a zip file.
3. run a cmd prompt as Administrator
4. type cd Desktop
5. type .\python-2.7.1.msi Click Run, Leave default Install for All Users, click Next, leave default Python27 destination, click Next, click Next, click Finish.
6. type .\pywin32-214.win32-py2.7.exe Click Next, Leave default of c:\Python27\ click Next, click Next, click Finish.
7. type .\Twisted-10.2.0.winxp32-py2.7.exe Click Next, Leave default of C:\Python27\, click Next, click Next, click Finish.
8. type .\setuptools-0.6c11.win32-py2.7.exe Click Next, Leave default of C:\Python27\, click Next, click Next, click Finish.
9. type c:\Python27\scripts\easy_install.exe zope.interface-3.6.1-py2.7-win32.egg
10. if you are installing a master, install Jinja2 using the easy_install method from the Jinja2 website.
11. if you are installing a master, install the correct version of pyOpenSSL from the 2.7 egg, from here https://launchpad.net/pyopenssl . This is necessary for SSL email to work (build failure emails).
12. if you are installing a master, unzip buildbot-0.8.5.zip in this folder. type cd buildbot-0.8.5\buildbot-0.8.5, type c:\python27\python.exe setup.py install
13. if you are installing a slave, unzip buildbot-slave-0.8.5.zip in this folder. type cd buildbot-slave-0.8.5\buildbot-slave-0.8.5, type c:\python27\python.exe setup.py install
14. reboot

Test Your Installation

Verify the slave installation. For example:

  C:\Documents and Settings\Administrator\Desktop>c:\python27\scripts\buildslave --version Buildslave version: 0.8.3 Twisted version: 10.2.0 

Create a User

The rest of this document assumes you are running as a non-privileged user. We recommend creating one specifically for your build slave.

Setup

Basic setup under windows is the same as under other operating systems, except that BuildBot currently cannot manage masters or slaves whose paths have spaces in them. If you absolutely must use a path with spaces, you can map that path to a drive letter and refer to it that way. Run buildslave create-slave to create your slave directory. Edit the info files in the slave directory as appropriate. Then start the buildslave process. Check twistd.log for any error messages.

You can call it quits and be done at this point. Any time the command prompt buildslave is running under closes, your slave will shutdown -- this includes reboots for windows update! Thus it is highly desirable to install buildbot as a windows service. When installed as a service, Windows will automatically restart your slave after a reboot.

Service

You'll need to grant your non-privileged user the ability to run services. To do this, run secpol.msc as an Administrator (the author found the easiest way to do this was to right-click a command line prompt and "run as administrator", and then enter secpol.msc in the command line). Then:

1. Select the "Local Policies" folder
2. Select the "User Rights Assignment" folder
3. Double click "Log on as a service"
4. Use "Add User or Group..." to add your non-privileged user here.

If you don't already have one open, open up a command shell running as administrator. Then use a builtin buildbot script to install a service for buildbot:

c:\python27\python.exe c:\python27\scripts\buildbot_service.py --user YOURDOMAIN\theusername --password thepassword --startup auto install 

(except all as one line, of course!) Note the "YOURDOMAIN\" part is required. For most setups, you can replace "YOURDOMAIN" with ".", making the full user name ".\theusername" (where theusername is the username of the non-privileged user, and thepassword is that user's login password). Ignore any messages about "Failed to register with the windows firewall".

The next bit of setup involves adding permissions on a certain registry key. However, the registry key does not yet exist because the service has never been started. Under Administrative Tools find Services and run it as an administrator. Then start the "Buildbot" service. Windows will immediately complain that the service shutdown immediately; that's fine.

In your administrator command prompt, run "regedit". Use it to navigate to "HKEY_LOCAL_MACHINE\System\CurrentControlSet\services"; in that long list of services you will find a "Buildbot" entry (i.e. folder). Right click the Buildbot folder and select permissions. In the dialog that comes up, add a new user to the list, and then grant that user "Full Control".

Now try again to start the service. Windows will again immediately complain that the service shutdown immediately; that's fine. This step is required in order to create the "Parameters" key we'll use in the next step.

Under that "Buildbot" services in the registry you will find a "Parameters" key (you may need to hit F5 to refresh so you can see it). There are currently no values under this key. You will need to *create* a 'String Value' under "Parameters" which is named "directories". Set the value of "directories" to be the full path to your slave's configuration directory. This is the same directory where buildbot.tac and twistd.log live.

You will now need to setup a buildbot master and/or slaves into the directory you specified. See Build Bot Tutorial: First Run for more information.

Finally, go back to your "Services" running as administrator and right-click "Buildbot" to start your service -- or just reboot.

Note: any environment variables that need to be set on the slave machine for your bot's build commands to work have to be set in the global System Environment settings or (better) in the unprivileged user's User Environment settings.

In a virtualenv

If you choose to run your service in a virtualenv, there are a few further issues you'll need to take care of.

First, PyWin32 won't install directly into a virtualenv, so you'll have to install it globally.

The service runs an executable installed by PyWin32 to c:\Python26\Lib\site-packages\win32\PythonService.exe. When you're doing the regedit steps above you'll see that path in the ImagePath key. PythonService.exe looks for python.exe in a folder relative to where it is installed, so you'll need to create some symlinks (or on pre-Vista systems, copies). Change that ImagePath to point into your virtualenv, e.g. C:\Users\buildslave\venv\Lib\site-packages\win32\PythonService.exe. Then symlink c:\Python26\Lib\site-packages\win32 into the corresponding place in your virtualenv. Finally, make a symlink for python, which is in your virtualenv's Scripts\ directory, in the root of the virtualenv.

Next, you'll want to update the service's environment so that the virtualenv is in the PATH. This article shows how to do that. Note: any environment variables set here will not propagate to the build commands issued by the bot. Those should be set in the User Environment of the unprivileged user as described above.

Many Buildslaves

Once you have mastered the basics of setting up a buildslave as a Win32 service, you might want to set up more buildslaves on the same system. Do not go to the trouble of cloning your BuildBot service - merely add your other slave configurations to the "directories" string value, separated by semicolons. For example:

directories   REG_SZ   C:\BuildBot\Proj1;C:\BuildBot\Proj2;C:\BuildBot\Proj3 

Consider using vms to manage your slaves. Create a clean base installation and clone it for each new build.

Restart the service and your new buildslaves will connect.

Disabling JIT dialogs on the buildslave

If you are running tests that occasionally fail with exceptions on the buildslave, you will want to disable the Just-In-Time (JIT) debugger that is configured with Visual Studio. When an unhandled exception occurs, the default JIT configuration will launch a dialog that will wait for user confirmation, effectively halting your build.

Follow these steps to disable the JIT:

1. Run regedit (as Administrator)
2. DELETE HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug\Debugger
3. DELETE HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\DbgManagedDebugger
4. SET HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\DbgJITDebugLaunchSetting to 0x1 (right click value, select Modify, hexadecimal, 1), ok
5. If you are running a 64 bit version of windows DELETE HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug\Debugger
6. If you are running a 64 bit version of windows DELETE HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\DbgManagedDebugger
7. Reboot

These settings are clarified at the following two sites. It should be noted that the registry changes from BOTH sites must be done.

http://msdn.microsoft.com/en-us/library/2ac5yxx6(v=VS.90).aspx

http://msdn.microsoft.com/en-us/library/5hs4b7a6(v=VS.90).aspx

Other Registry Settings

You will probably want to disable the Windows Error Reporting facility on Windows 7. This will prevet the annoying dialog box of "Report this error". Under HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\Windows Error Reporting add a REG_DWORD of "Disabled" and set it to "1"

Troubleshooting ¶

If you have any issues with the service installation or buildbot-as-a-service startup, you can see the actual problem using Windows' "Event Viewer". It's under "Windows Logs" and then "Applications"; look for problems of type "Error" with a source of "Buildbot". Please report any such errors when asking for help on the mailing list.

The buildbot master.cfg needs the slave information; the master will ignore a slave it doesn't know about.

If the buildbot service fails to start, but the script runs for the same user when logged in, it may be due to permissions that were in effect when the buildbot package and pre-requisites were installed. See "PreRequisites?" above.

Visual Studio build configs such as "Win32|Any CPU" will need to be escaped thus: "Win32^|Any CPU". Consider saving yourself development time by using python instead of bat and cmd for slave scripts.