This page contains step by step instructions to setup a development server with Subversion, Trac,
Jenkins. This is how I usually setup a development server on an Ubuntu Server. I hope you find it useful.
Please note that some parts might be aged, or may not suit to your needs. If you have a question or
a comment, you can always reach me at fatih[at]fmguler.com
And don't forget to change johndoe with your username ;)
1.Initial User Management
After OS setup, first add a new user, restrict sudoers to admin group, create admin group, and add this user to admin group in order to make it a sudoer, finally disable root;
adduser johndoe vi /etc/pam.d/common-password //can set min=8 for security visudo %admin ALL=(ALL) ALL //add this line under user privilidge spec. groupadd admin adduser johndoe admin //add the new user to admin group which will make it sudoer //login as johndoe sudo passwd -l root //disable root account sudo dpkg-reconfigure tzdata //set time zone to your place
Next, restrict ssh access to sshlogin group and add this new user to that group;
//restrict ssh login to sshlogin group users sudo vi /etc/ssh/sshd_config //add these to end #restrict ssh access AllowGroups sshlogin //add user johndoe to sshlogin group so that we can login with ssh sudo groupadd sshlogin sudo adduser johndoe sshlogin sudo /etc/init.d/ssh restart
Then add a restricted user: devuser which will run all restricted applications. Note that this user cannot connect with ssh, and owns all the application folders. If you need to modify the files within /development you need to change user to this (sudo su devuser).
sudo adduser devuser
REMINDER: You can install applications or do root privilidged operations with the sudoer user (johndoe or another sudoer). You can modify files in /development with the restricted user (devuser).
2. Install Dropbox
Install Dropbox, with the restricted user; (http://wiki.dropbox.com/TipsAndTricks/TextBasedLinuxInstall)
//login as devuser or change user to devuser (sudo su devuser) cd /home/devuser wget -O dropbox.tar.gz http://www.dropbox.com/download\?plat=lnx.x86 tar -xzvf dropbox.tar.gz cd .dropbox-dist/ ./dropboxd
Exit dropboxd. Create the /development folder layout (link to dropbox folders);
//as a sudoer (e.g. johndoe) cd / sudo mkdir development sudo chown devuser:devuser development/ sudo su devuser cd development //link dropbox folders to /development counterparts ln -s /home/devuser/Dropbox/somefolder/bin/ bin ln -s /home/devuser/Dropbox/somefolder/backup/ backup ln -s /home/devuser/Dropbox/somefolder/meta/ meta
Disable dropbox lan sync. feature to prevent listen port 17500;
cd /development/bin chmod +x *.py chmod +x *.sh ./dropbox.py stop ./dropboxp2p.py -d ./dropbox.py start
NOTE: You can find dropbox.py here.
3. About Folder Layout
The folder layout of the development infrastructure is like below;
- /development: root folder
- /development/bin: binary folder, contains start and backup scripts.
- /development/backup: backup folder, where the backups are taken to.
- /development/meta: configuration file templates and related infrastructure files.
- /development/svn: subversion root folder, contains svn repos.
- /development/trac: trac installation folder, contains trac binary and trac projects under scripts folder.
- /development/hudson: hudson root folder, contains hudson jobs and binaries.
- /development/jetty: jetty root folder, contains java web applications.
The list of programs and services utilized;
Below are the setup process for each of them;
- dropbox: already defined above. It is used for backup and central storage of scripts and configuration templates.
- subversion: install subversion package with apt-get;
sudo apt-get install subversion sudo su devuser cd /development mkdir svnThen copy subversion repositories to this svn folder from backup (as devuser user). Later, test subversion by starting the svn start script: /development/bin/start-svn.sh. You may need to change listen address from this script to 127.0.0.1 since openvpn is not installed yet. Forward 3690 port with putty. Test with repo browser. After testing close it, because we will start this process by another start script.
- trac: to install trac follow these steps;
sudo apt-get install python-setuptools sudo apt-get install python-subversion sudo apt-get install language-pack-tr sudo easy_install Trac==0.12 sudo easy_install http://trac-hacks.org/svn/tracwysiwygplugin/0.12 sudo su devuser cd /development mkdir tracThen copy trac installations to this trac folder from backup (as devuser user). Later, test trac by starting the trac start script: /development/bin/start-trac.sh. Forward 8000 port with putty. Test with browser. After testing close it, because we will start this process by another start script. While testing if it says this, do it so: The Trac Environment needs to be upgraded. Run "trac-admin /development/trac/YourProject upgrade"
install account manager plugin : account manager plugin makes it easy to manage user accounts. To install it do;
sudo easy_install http://trac-hacks.org/svn/accountmanagerplugin/trunk
Then enable these modules from trac admin page: AccountManagerAdminPage , AccountManager , HtDigestStore , HtDigestHashMethod , AccountModule , LoginModule
Disable LoginModule from trac components. From Accounts>Configuration menu make filename: /development/trac/users.htdigest and realm: trac then save. trac-start.sh shouldn't have the --auth part. Restart trac.
trac post commit hook : used to give auto reference from post messages to trac tickets;
cp post-commit.tmpl post-commit chmod 755 post-commit vi post-commit /usr/local/bin/trac-admin /development/trac/xyz changeset added "$1" "$2" //if you migrate from windows you shoud do in vim; :set ff=unix
Then enable tracopt.ticket.commit_updater.* under trac components from admin page.
trac notifications & email : to enable mail sending from trac, for notifications and password resetting, change smtp address to 127.0.0.1 from trac.ini and do;
sudo apt-get install postfix sudo dpkg-reconfigure postfix //select internet site from the list //now can test mailing with; sudo apt-get install bsd-mailx echo test | mail -s "test mail sent to external" [email protected] //and check mail by typing; mail //forwarding incoming mail can be done with; http://www.cyberciti.biz/faq/linux-unix-bsd-postfix-forward-email-to-another-account/
- hudson: Hudson is the build server used to
build projects and store binaries. It is a Continuous Integration (CI)
tool which builds the project on every commit. The steps to install and
cd /development/hudson wget http://hudson-ci.org/latest/hudson.war sudo apt-get install openjdk-6-jre-headless //can start hudson by the script or java -jar hudson.war /development/bin/start-hudson.shAfter that, install required programs to /development/hudson/prog (or somewhere else) and configure paths inside hudson admin web interface.
cd /development/hudson/prog/setup wget http://.../jdk-6u21-linux-i586.bin wget http://.../jdk-1_5_0_22-linux-i586.bin wget http://download.oracle.com/otn-pub/java/j2sdk/1.4.2_19//j2sdk-1_4_2_19-linux-i586.bin wget http://mirror.veriportal.com/apache//ant/binaries/apache-ant-1.8.1-bin.zip //then unzip/run these, and move extracted folders to /development/hudson/prog cd /usr/bin sudo ln -s /development/hudson/prog/apache-ant-1.8.1/bin/ant antFor .NET and Win32 projects the mono alternative clearly sucks. Do not waste your time trying to build your .NET projects with mono and xbuild. You cannot build Win32 projects anyway. Instead you can run .NET and Win32 projects in a windows Hudson slave. For this, set up a windows machine (virtualbox, or vmware is fine), install jre, and login to hudson (master hudson, e.g. hudson.example.com) from this machine. Then add a slave node from manage hudson -> slaves. Finally, run jnlp slave agent, and that's it. Now you can set from the job configuration "Restrict where this project can be run" to the slave's name, and these jobs will run at the slave. You shoud install .net sdk, msbee to the slave machine (the easiest is to install visual studio). The latest quirk is to remove msbuild configuration from manage hudson, so that hudson will not look for msbuild.exe on the linux master machine. To start java jnlp slave agent from command line, the command is;
java -Duser.language=en -jar slave.jar -jnlpUrl http://hudson.example.com/computer/hudsonwin/slave-agent.jnlp
- openvpn: Open VPN is used to
access services which are not exposed to public IP, instead run on local
IP's. Follow these steps to install; (https://help.ubuntu.com/10.04/serverguide/C/openvpn.html)
sudo apt-get install openvpn //init CA sudo mkdir /etc/openvpn/easy-rsa/ sudo cp -r /usr/share/doc/openvpn/examples/easy-rsa/2.0/* /etc/openvpn/easy-rsa/ sudo chown -R johndoe /etc/openvpn/easy-rsa/ cd /etc/openvpn/easy-rsa/ vi vars //edit KEY_COUNTRY, KEY_CITY, etc at the bottom. source vars ./clean-all ./build-dh ./pkitool --initca ./pkitool --server "John Doe Development Server" cd keys openvpn --genkey --secret ta.key sudo cp John\ Doe\ Development\ Server.crt John\ Doe\ Development\ Server.key ca.crt dh1024.pem ta.key /etc/openvpn cd ../../ /rename John Doe Development.* to server.* //edit server conf sudo cp /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz /etc/openvpn/ sudo gzip -d /etc/openvpn/server.conf.gz sudo vi /etc/openvpn/server.conf sudo /etc/init.d/openvpn restart
client configuration : First generate keys for the client;
cd /etc/openvpn/easy-rsa/ source vars ./pkitool hostname //copy these files to client's ; /etc/openvpn/ca.crt /etc/openvpn/easy-rsa/keys/hostname.crt /etc/openvpn/easy-rsa/keys/hostname.key /etc/openvpn/ta.key
In the client install open vpn client for windows (http://openvpn.net/index.php/open-source/downloads.html ) and put the files specified above to: C:\Program Files\OpenVPN\config. Then copy client.ovpn from sample-config folder in the client to the config folder. Edit it's server address (remote dev.yoursite.com 1194). Run OpenVPN GUI as administrator, click connect.
site to site vpn : To access the development server private services everyone must install openvpn client. To prevent this, we can make site to site vpn and route all traffic to the vpn site from a single vpn client. In this scenario, we have a ubuntu server box in the local area network which will be the vpn box. Just insall openvpn with "sudo apt-get install openvpn". Then copy the client configuration files to this server. After connecting as a regular client, we will route all the traffic to the vpn network from this client. In order to forward traffic we must enable packet forwarding. Here are all the steps to establish site to site vpn;
//In the ubuntu server vpn box; sudo apt-get install openvpn cd /home/johndoe mkdir openvpn-10.8.0.0 cd openvpn-10.8.0.0 //download here vpn client files from My Dropbox » dev.yoursite.com » meta » conf-templates » openvpn-client sudo openvpn --config client.conf //In another screen (assuming you are working in screen, use ctrl+c) sudo vi /etc/sysctl.conf //uncomment this line to enable packet forwarding net.ipv4.ip_forward=1 sudo su echo 1 > /proc/sys/net/ipv4/ip_forward exit //packet forwarding done
After connecting from a single client and enabling packet forwarding, we can connect from other machines through the vpn machine. But the last problem is that our gateway should know to route 10.8.0.0 traffic through 10.0.0.x machine. Either do that from the firewall or add this to the connecting machines (assuming windows);
//In the windows machines add this route: route add 10.8.0.0 MASK 255.255.255.0 10.0.0.52
- nginx: Nginx is a lightweight web server and
reverse proxy server. We use it as a reverse proxy server to redirect
multiple sub domains to various web applications. The steps to install
nginx and configure it to proxy trac.yoursite.com to 127.0.0.1:8000 are;
sudo apt-get install nginx cd /etc/nginx/sites-available sudo vi default //to return 444 for any unknown domain //change server name to _ and add return 444 t server_name _; return 444; //exit with :x //copy the trac configuration file from meta folder sudo cp /development/meta/conf-templates/nginx\ \(etc-nginx-sites-available\)/trac.yoursite.com ./ cd ../sites-enabled/ sudo ln -s /etc/nginx/sites-available/trac.yoursite.com trac.yoursite.com sudo /etc/init.d/nginx restart //now you can access to trac as http://trac.yoursite.com/ //copy the hudson configuration file from meta folder sudo cp /development/meta/conf-templates/nginx\ \(etc-nginx-sites-available\)/jenkins.yoursite.com ./ cd ../sites-enabled/ sudo ln -s /etc/nginx/sites-available/jenkins.yoursite.com jenkins.yoursite.com sudo /etc/init.d/nginx restart //now you can access to hudson as http://jenkins.yoursite.com/
- jetty: Jetty is an embedded
java servlet container. It is preferred over tomcat because of easy
management and lower memory consumption. Java web applications (.war
files) are deployed to jetty. The steps to install jetty are;
wget http://download.eclipse.org/jetty/stable-7/dist/jetty-distribution-7.2.0.v20101020.tar.gz tar xzvf jetty-distribution-7.2.0.v20101020.tar.gz java -Xmx100m -jar start.jar //edit etc/jetty.xml to change listening port or thread pool //also you can configure nginx to forward external traffic (same as trac/hudson)
5. Firewall (Important!!)
Since none of the ports of linode servers are firewalled, we have to (we'd better :)) install a firewall, since we might misfollow the instructions and not close some listening ports. For this, the command which lists the listening ports is our precious friend;
sudo netstat -antp | grep LISTEN
Here 0.0.0.0:something is very dangerous. If you do not understand these at all, do yourself a favor and install a firewall;
sudo apt-get install ufw //add all of the ports to be served publicly sudo ufw allow 22 sudo ufw allow 80 sudo ufw allow 443 sudo ufw status //still inactive sudo ufw enable sudo ufw status verbose
6. Fail2Ban (SSH Brute Force Security)
It is hard to force everyone to connect to ssh with keys (but must be done some day, if that day ever comes, just edit /etc/ssh/sshd_config and turn off password authentication like: PasswordAuthentication no). Until then, it is easier to install fail2ban which will ban brute force IP's (3 failed ssh attempts will be banned for 10 minutes). To install fail2ban;
sudo apt-get install fail2ban cd /etc/fail2ban sudo cp jail.conf jail.local sudo vi jail.local //change destemail to [email protected] //change action = %(action_)s to action = %(action_mw)s //change ssh maxretry to 3 //save sudo vi /etc/rsyslog.conf //change $RepeatedMsgReduction off //save sudo /etc/init.d/rsyslog restart sudo /etc/init.d/fail2ban reload
Then try to enter with invalid passwords. Fail2ban should send email with whois information to the specified email and ban you for 10 minutes.
Too many fail2ban emails? Add allowed ssh IPs to hosts.allow;
sudo vi /etc/hosts.allow //add> sshd: X.Y.Z.D, A.B.C.* sudo vi /etc/hosts.deny //add> sshd: ALL