-
Notifications
You must be signed in to change notification settings - Fork 0
KVMAutotest GetStartedServer
We have developed a script to automate the steps described below on a (Fedora 16/17/RHEL6.2) server. On an autotest source clone, this script is under contrib/install-autotest-server.sh:
https://github.com/autotest/autotest/blob/master/contrib/install-autotest-server.sh
If you want to download it straight from github, just
curl https://raw.github.com/autotest/autotest/master/contrib/install-autotest-server.sh > install-autotest-server.sh
Then make it executable and execute it:
chmod +x install-autotest-server.sh ./install-autotest-server.sh
Note: If the distro you are running has Django >= 1.3 packaged (such as Fedora 15 onwards), you can install the django that your distro ships:
yum install Django
Otherwise, it's best to leave to build_externals.py the task of installing
it. Other needed packages:
yum install git make wget python-devel unzip yum install httpd mod_wsgi mysql-server MySQL-python gnuplot python-crypto python-paramiko java-1.6.0-openjdk-devel python-httplib2 yum install numpy python-matplotlib libpng-devel freetype-devel python-imaging
Important: For this entire documentation, we will assume that you'll install autotest under /usr/local/autotest. If you use a different path, please change /usr/local/autotest accordingly. Please that you may have some issues with apache configuration if you don't choose /usr/local/autotest.
Important: We will also assume that you have created an autotest user on your box, that you'll use to perform most of the instructions after the point you have created it. Most of the instructions will use autotest unless otherwise noted.
As root:
useradd autotest passwd autotest [type in new password]
You can then clone the autotest repo (as root):
cd /usr/local git clone git://github.com/autotest/autotest.git chown -R autotest:autotest autotest
Log out, re-log as autotest, and then proceed.
The way the kvm control file is organized right now requires the user to change one value on global_config.ini file, that should be at the top of the autotest tree.
As autotest, please change the following configuration value from what's default to make it look like this:
[PACKAGES] serve_packages_from_autoserv: False
By default, the above value is True. To make a long story short, changing this value will make autoserv to copy all tests to the server before trying to execute the control file, and this is necessary for the kvm control file to run. Also, we need the other tests present to run autotest tests inside guests.
Make sure that mysql daemon is up and starts on each boot. As root:
/sbin/service mysqld restart chkconfig mysqld on
Next,provide mysql server with password by running the following command (as autotest or root, you choose):
mysqladmin -u root password YOUR_PASSWORD
Where YOUR_PASSWORD is your administrative password for mysql. To get a
mysql query prompt, type mysql -u root -p. The following commands
will set up mysql with a read-only user called nobody and a user with
full permissions called autotest with a password 'some_password', and
must be typed on mysql's query prompt:
create database autotest_web; grant all privileges on autotest_web.* TO 'autotest'@'localhost' identified by 'some_password'; grant SELECT on autotest_web.* TO 'nobody'@'%'; grant SELECT on autotest_web.* TO 'nobody'@'localhost'; create database tko; grant all privileges on tko.* TO 'autotest'@'localhost' identified by 'some_password'; grant SELECT on tko.* TO 'nobody'@'%'; grant SELECT on tko.* TO 'nobody'@'localhost';
Run the build script to install necessary external packages (You will need unzip to be able to use setuptools). Currently, the packages it installs are setuptools, mysql-python, Django, numpy, matplotlib, paramiko, simplejson and GWT. As autotest:
/usr/local/autotest/utils/build_externals.py
Always re-run this after a svn update/git pull if you notice it has changed, new dependencies may have been added. This is safe to rerun as many times as you want. It will only fetch and build what it doesn't already have. It's important to note that the autotest scheduler will also try to run build_externals.py whenever it's executed in order to make sure every piece of software has the right versions.
Important: Set the HTTP_PROXY environment variable to
http://proxy:3128/ before running the above if
your site requires a proxy to fetch urls.
As root:
ln -s /usr/local/autotest/apache/conf/all-directives /etc/httpd/conf.d/autotest.conf
Test your configuration (now with all autotest directives) by running (as root):
service httpd configtest
Important: Edit the following files to match the database passwords you set earlier during session #Set_up_MySQL, as autotest.
/usr/local/autotest/global_config.ini /usr/local/autotest/shadow_config.ini
Things that you usually want to change on global_config.ini:
In addition, change the notify_email_from field. You might also use a dedicated smtp server, without relying on the one you have running on the machine, check in the config file the options available.
During the time span of the project, the autotest database went through design changes. In order to make it able for people running older versions to upgrade their databases, we have the concept of migration. Migration is nothing but starting from the initial database design until the latest one used by this specific version of the application. As autotest:
/usr/local/autotest/database/migrate.py --database=AUTOTEST_WEB sync
You have to run syncdb twice, due to peculiarities of the way syncdb works on Django. As autotest:
/usr/local/autotest/frontend/manage.py syncdb /usr/local/autotest/frontend/manage.py syncdb
Compile the Autotest web application and TKO frontend. As autotest:
/usr/local/autotest/utils/compile_gwt_clients.py -a
You will need to re-compile after any changes/syncs of the frontend/client pages.
You may encounter issues with SELinux not allowing a section of the web UI to work when running in Enforcing Mode. In order to fix this, you can run the following commands to allow execution of the cgi scripts on your server.
As root:
semanage fcontext -a -t httpd_sys_script_exec_t '/usr/local/autotest/tko(/.*cgi)?' restorecon -Rvv /usr/local/autotest
Note: If you are having weird problems with installing autotest, you might want to turn off SElinux to see if the problem is related to it, and then think of a sensible solution to resolve it.
As root:
/sbin/service httpd restart
You should be able to access the web frontend at http://localhost/afe/, or http://your.server.fully.qualified.name.or.ip/afe/
Updated: Added Red Hat style init scripts and systemd service files, for Fedora15 onwards.
As root:
cp /usr/local/autotest/utils/autotest-rh.init /etc/init.d/autotestd chkconfig --add /etc/init.d/autotestd service autotestd start
Copy the service file to systemd service directory. As root or using sudo:
sudo cp /usr/local/autotest/utils/autotestd.service /etc/systemd/system/
Make systemd aware of it:
sudo systemctl daemon-reload
Start the service:
sudo systemctl start autotestd.service
Check its status:
autotestd.service - Autotest scheduler
Loaded: loaded (/etc/systemd/system/autotestd.service)
Active: active (running) since Wed, 25 May 2011 16:13:31 -0300; 57s ago
Main PID: 1962 (autotest-schedu)
CGroup: name=systemd:/system/autotestd.service
├ 1962 /usr/bin/python -u /usr/local/autotest/scheduler/autotest-scheduler-watcher
└ 1963 /usr/bin/python -u /usr/local/autotest/scheduler/autotest-scheduler /usr/local/autotest/results
You can execute the babysitter scripter through, let's say, nohup or screen. It is important to remember that by design, it's better to create an 'autotest' user that can run the scheduler and communicate with the machines through ssh. As root:
yum install screen
As autotest:
screen /usr/local/autotest/scheduler/monitor_db_babysitter
You can even close the terminal window with screen running, it will keep the babysitter process alive. In order to troubleshoot problems, you can pick up the log file that monitor_db_babysitter prints and follow it with tail. This way you might know what happened with a particular scheduler instance.
You can find them in the autotest logs directory. As autotest or root:
tail -f /usr/local/autotest/logs/scheduler-[timestamp].log
Run /usr/local/autotest/client/tests/kvm/get_started.py as autotest
to be guided through the process of setting up the autotest config files.
Edit the files to suit your testing needs.
The server is now ready to use. Please check out the following sections to learn how to configure remote hosts and execute the KVM test suite.
Clients are managed in the tab hosts of the web frontend. It is important that you can log onto your clients from your server using ssh without requiring a password.
As autotest, on the server, create a DSA key in the following way:
ssh-keygen -t dsa
Then, still on the server, and as autotest, copy it to the host:
ssh-copy-id [email protected]
You can import all the available tests inside the autotest client dir by running the test importer script as autotest:
/usr/local/autotest/utils/test_importer.py -A
The KVM test will be among the imported tests.
It is recommended to work with the new results interface provided by autotest allowing SQL query based filtering, usable display of logs and test attributes and graphing capabilities. However, kvm-autotest also produces a detailed, formatted html report (job_report.html) per remote host tested in addition to standard autotest logs, where kvm-autotest results data is nicely organized. The html reports are stored in the job main results directory (accessible via raw results logs link).
You need to re-run scheduler. You are strongly advised to use the init scripts mentioned above, and restart the service, in case it's not running, you can execute as root.
service autotestd start
Usually it is a result of scheduler crash due to lack of disk space on Autotest server.