Mercurial Helper Script

Explanation

This is the current version of the mercurial-helper script that I have created. The general purpose of this script is to aid users in creating and adding users to the password file hgusers (typically called that), as well as setting up a repository.

Unfortunately I have not added any functionality to remove users from the password file, though this is not a difficult task to carry out. Users should be able to change/update their passwords in the hgusers file just by re-entering their credentials. A second problem is that script requires sudo access to make sure that the directories and permissions can be altered correctly.

How to Use

Acquire the script by creating a file called mercurial-helper.sh and pasting in the following section's code into it. Then place this file where the mercurial repositories are going to be found (if the earlier blog entries were followed that would be /srv/redmine/extra/hg/). Now to execute it:
# sudo sh mercurial-script.sh

Then the script should guide you through the rest of the steps to create and add users to the password file, as well as creating a repository.

mercurial-script.sh

#!/bin/bash
#-----------------------
# Author: Kevin Jalbert
# Date: June 6th, 2010
#-----------------------

# Variables that are used
choice=4
name="----"
passwordFile="----"
realm="----"
repository="----"

# Present the user with potential options
echo "1. Add/Change User"
echo "2. Create New Repository"
echo "3. Exit"
echo -n "Please choose an option [1,2, or 3]? "

# Keep looping till a valid choice is made
while [ $choice -eq 4 ]; do

 read choice

 # Adding/changing a user
 if [ $choice -eq 1 ] ; then

  echo "--Adding/changing a user--"

  # Acquire the name
  echo "Enter the user's account name"
  read name

  # Acquire repository's realm
  echo "Enter the repository's realm (should be mercurial)"
  read realm

  # Acquire file's name
  echo "Enter name of password file (should be hgusers)"
  read passwordFile

  # Check to see if this file already exists, and perform appropriate action
  if [ -e $passwordFile ]; then
   echo `htdigest $passwordFile $realm $name`
  else
   echo `htdigest -c $passwordFile $realm $name`
  fi

  echo "User [$name] has been added/changed within password file [$passwordFile]"

 # Creating a new repository
 elif [ $choice -eq 2 ] ; then

  echo "--Creating a new repository--"

  # Acquire the repository's name
  echo "Enter the repository's name"
  read repository

  # Acquire the users involved in this repository
  echo "Enter the users involved in this repository (Comma Separated Values)"
  read name

  # Set up the repository
  echo `hg init $repository`
  echo `touch ./$repository/.hg/hgrc`
  echo "[paths]
default = $PWD/$repository/

[web]
allow_push = $name
contact = $name
description = Fill me out (in ./project/.hg/hgrc)
push_ssl = false
style = gitweb

[trusted]
users = *
groups = *

[hooks]
changegroup = hg update" > ./$repository/.hg/hgrc

  # Change the ownership permissions
  echo `chown -R www-data ./$repository/`
  echo `chmod -R 755 ./$repository`

  echo "The repository is created. The configuration file located at ./$repository/.hg/hgrc
  needs to be edited to ensure that the path to this repository is set as well as any
  additional changes that are desired."

 # Exiting
 elif [ $choice -eq 3 ] ; then

  echo "Exiting"
  exit;

 # Script input mismatch
 else

  $choice=4
  echo "Incorrect input, try again"
  echo "1. Add/Change User"
  echo "2. Create new Repository"
  echo "3. Exit"
  echo -n "Please choose an option [1,2 or 3]? "

 fi
done

Mercurial Setup Guide

1 Introduction

Repositories are useful in managing software projects, mainly because they provide a means of source code collaboration. There are various forms of repositories, though this post will talk about Mercurial, which is a distributed version control system. Mercurial is typically used in situations where a software project is managed in a team, with various users developing on the same code base. A source control management system can be internal which means that it is only available on the local machine or local area network. The internal solution is ideal for personal projects work, though for more open work an external solution is necessary to allow for accessibility from any Internet connection. This guide will detail the process of setting up an external solution since it requires much more work then an internal set up. To do this Mercurial will be set up behind an Apache HTTP Server that is configured to provide external access.

2 System Setup

Due to the popularity of Ubuntu, and personal usage of Arch Linux both commands will be outlined when needed. The commands will begin with either ubuntu# or arch# for the specific distribution, otherwise just a # will be used when both are applicable. In addition, most Linux distributions are fairly similar which means that the installations of packages will be the same for the majority. The release for Arch Linux is 2009.08, and the release for Ubuntu is 10.04, both on 32-bit architecture.

For Ubuntu, just perform the following to acquire and setup Apache, PHP and MySQL. Then skip to section 3.
ubuntu# sudo apt-get install lamp-server^

2.1 Install Apache

Install the Apache package.
arch# sudo pacman -S apache

Apache should work right now, a quick test using the following command should start Apache.
arch# sudo /etc/rc.d/httpd start

To confirm it works browse to http://localhost/. Then close down Apache for now.
arch# sudo /etc/rc.d/httpd stop

Make the Apache service starts on boot, the httpd deamon will be added to the /etc/rc.conf file.
DEAMONS=(... httpd ...)

2.2 Install PHP

Install the required PHP packages.
arch# sudo pacman -S php php-apache

Add the following to the /etc/httpd/conf/httpd.conf file at the end of the LoadModule list.
LoadModule php5_module modules/libphp5.so

Add the following to the /etc/httpd/conf/httpd.conf file at the end of it.
Include conf/extra/php5_module.conf

Change the following line in the /etc/httpd/conf/httpd.conf file to include the bold word.
Options Indexes FollowSymLinks Includes

To test that PHP is operational first make a test file /srv/http/test.php with the following text.
<? php phpinfo(); ?>

Restart/start Apache back up.
arch# sudo /etc/rc.d/httpd start

To confirm it works browse to http://localhost/test.php and PHP information should be visible.

3 Mercurial Repositories

This section is dedicated to making Mercurial repositories accessible externally from the server. The first thing is to first install the Mercurial packages.
ubuntu# sudo apt-get install mercurial
--or--
arch# sudo pacman -S mercurial

3.1 Create Folder for Mercurial Repositories

A directory that is not located directly in the web server needs to be created to hold just the repositories (done so that repositories are not exposed). For me Redmine is being used, so ideally the repositories can be placed in there. The following directory needs to be created:
# sudo mkdir /srv/redmine/extra/hg

3.2 Create Folder for Mercurial Scripts

A directory in the web server folder needs to be created to hold the CGI scripts to allow mercurial to be accessible via the web server. The following directory needs to be created:
arch# sudo mkdir /srv/http/cgi-bin
--or--
ubuntu# sudo mkdir /var/www/cgi-bin

3.3 Create hgweb.config File

Within the previously created folder cgi-bin, the file hgweb.config needs to be created. This file directs where the Mercurial repositories are located. It needs to be created with the following contents:

[collections]
 /srv/redmine/extra/hg/ = /srv/redmine/extra/hg/

The created file is the bare-minimum required for this set up to function. The paths should be set correctly to the system's file structure. In the case where a more fine-tuned file is needed, please consult the mercurial support page.

3.4 Create hgwebdir.cgi File

Within the previously created folder cgi-bin, the file hgwebdir.cgi needs to be created. This file is used to enable interaction from the external connection on the Mercurial repositories. It needs to be created with the following contents:

#!/usr/bin/env python
 from mercurial import demandimport; demandimport.enable()

 import cgitb
 cgitb.enable()

 import os
 os.environ["HGENCODING"] = "UTF-8"

 from mercurial.hgweb.hgwebdir_mod import hgwebdir
 import mercurial.hgweb.wsgicgi as wsgicgi

 application = hgwebdir('hgweb.config')
 wsgicgi.launch(application)

The created file is the bare-minimum required for this set up to function. In the case where a more fine-tuned file is needed, please consult the Mercurial support page.

3.5 Configure Permissions

The web server need to have executable permissions on the hgwebdir.cgi file to allow for proper functionality. This can be set by being in the correct directory and executing the following command in terminal:
# sudo chmod a+x /path/to/hgwebdir.cgi

3.6 Configure Apache Access to the Repositories

A file is needed to configure Apache to direct certain HTTP requests to act towards the mercurial repositories. This is done by editing/adding the following to the httpd.conf, the file is located at:
arch# /etc/httpd/conf/httpd.conf
--or--
ubuntu# /etc/apache2/httpd.conf

The following contents is what should be included in the file (note the paths need to be configured according to the system):

ScriptAliasMatch        ^/hg(.*)        /path/to/cgi-bin/hgwebdir.cgi$1

<Directory /path/to/cgi-bin/>
   Options ExecCGI FollowSymLinks
   AllowOverride None
</Directory>

<Location /hg>
  AuthType Digest
  AuthName "mercurial"
  AuthDigestProvider file
  AuthUserFile /srv/redmine/extra/hg/hgusers
  Require valid-user
</Location>

The idea is that for security reasons the authentication type used will be digest and it will only transmit the MD5 hash of the passwords. The authentication name must be consistent with the generation of the password(the realm), and acts as an additional salt value (pick what ever you will remember, since you will need it for every user you will create). The authentication file will be located beside the repositories and in this example is called hgusers. This file needs to be created and managed using the htdigest command (a script to simplify this is attached to this guide). This configuration also will only allow for the repositories to be accessed if the user is authenticated. For additional configurations please refer to the Mercurial website's instructions.

In addition, due to the use of the digest authentication method Apache requires the appropriate modules. For Arch Linux the modules should already be enabled by default, if not by chance you just need to enable the following modules:
auth_digest

In Ubuntu that module must be moved from the /etc/apache2/mods-available/ to /etc/apache2/mods-enabled/

4 Services Available

There are now a few services available on this server:

• Functioning Apache web server
  
• PHP and MySQL enabled on the web server
  
• Protected Mercurial repositories available externally
  

With these services it become possible to host a website to indicate the location of the Mercurial repositories or possibly to a different tool (such as Redmine). MySQL is a useful service to have available regardless since there are various applications out there that can benefit from a standard database such as MySQL.

Mercurial now exists on the system and so users are able to perform the basic Mercurial functions. Cloning from any Mercurial repository, as well as being able to start local repositories are now possible. In addition there is always the possible of creating an hosting externally accessible repositories such as how this guide as instructed.

5 Conclusion

By the end of this guide Mercurial should be completely set up along with Apache and PHP. Repositories are externally available and a greater understanding on how to manage it should have been acquired. If in doubt there is plenty of resources available online. There is also the mercurial-helper.sh script that I made to help with user and repository management (I will post about this script eventually).

5.1 Key Notes

For the remote repositories to automatically update (using the hooks) the owner of the repository must be what runs the web service. In most cases this is the www-data user, if ever in doubt you can always use these commands to reset the owner and modes.
# sudo chown -R www-data /srv/redmine/extra/hg/
--then--
# sudo chmod -R 755 /srv/redmine/extra/hg/

Additionally, a typical ./project/.hg/hgrc configuration would looks like:

[paths]
default = /srv/redmine/extra/hg/project/

[web]
allow_push = user
contact = user
description = Fill me out (in ./project/.hg/hgrc)
push_ssl = false
style = gitweb

[trusted]
users = *
groups = *

[hooks]
changegroup = hg update

These settings are used to allow users (which have to be authenticated using the hgusers file) to push back to the externally located repository. The reasoning for the hook section is to have the external repository to update when it receives a change set from a user. The trusted section is used to ensure that regardless of the users and groups using the repository (locally) that the pushes will still carry through. The rest of these settings can be found in the Mercurial documentation.

Redmine 0.9.4 Setup Guide

1 Introduction

Redmine is an open-source collaborate project management solution. Redmine operates as a web service that allows for multiple projects to be hosted in a single location. The goal of this guide is to quickly and efficiently set up Redmine on a server that can be used immediately.

2 System Setup

Due to the popularity of Ubuntu, and personal usage of Arch Linux both commands will be outlined when needed. The commands will begin with either ubuntu# or arch# for the specific distribution, otherwise just a # will be used when both are applicable (also mysql# will appear briefly when dealing with MySQL). In addition, most Linux distributions are fairly similar which means that the installations of packages will be the same for the majority. The release for Arch Linux is 2009.08, and the release for Ubuntu is 10.04, both on 32-bit architectures.

2.1 Install Ruby

Redmine requires Ruby to execute the code, as well as to run.
ubuntu# sudo apt-get install ruby rubygems libopenssl-ruby1.8 rake
--or--
arch# sudo pacman -S ruby rubygems

2.2 Install Subversion

To acquire the Redmine source code Subversion will be used. (Could very well use another source control management tool like Git)
ubuntu# sudo apt-get install subversion
--or--
arch# sudo pacman -S subversion

2.3 Install MySQL

Redmine makes use of a database to retain all its information, and thus requires one. MySQL will be chosen though you could use another database if you desire. Proceed with any prompts to initialize MySQL.
ubuntu# sudo apt-get install mysql-server libmysql-ruby
--or--
arch# sudo pacman -S mysql

2.4 Install Ruby Gems

Ruby has the concepts of gems which is very similar to packages to Linux. Gems contain additional functionality for the Ruby language. Ruby on Rails is used to allow Redmine to act as a web server. First the 2.3.5 version of the rails gem needs to be installed for Redmine to work, then the other specific gems.
# sudo gem install -v=2.3.5 rails
--then--
ubuntu# sudo gem install rake rack
--or--
arch# sudo gem install rake rack mysql

2.5 Install Thin

Ruby on Rails is a decent web server for Ruby web applications. Thin is an alternative to the original server(WEBrick), and offers more performance due to its lightweight nature.
ubuntu# sudo apt-get install thin
--or--
arch# sudo gem install thin

3 Redmine Setup

Now that all the packages have been acquired, the actual process of setting up Redmine can start.

3.1 Acquire Redmine Source Code

The Redmine source code is required to progress any further. Redmine's version 0.9.4 will be used as it is the latest stable release. Use the following command preferably in the /srv/ directory (best practices on where to place server material).
# sudo svn checkout http://redmine.rubyforge.org/svn/tags/0.9.4 redmine

3.2 Create Redmine Database

First ensure that MySQL is set up securely by using the following command.
# sudo mysql_secure_installation

Now to enter MySQL.
# sudo mysql -u root -p

Now create the redmine database.
mysql# create database redmine character set utf8;

Now create a redmine user for this database, with a of your choice.
mysql# create user 'redmine'@'localhost' identified by '';

Now to give this new redmine user all privileges on the database.
mysql# grant all privileges on redmine.* to 'redmine'@'localhost';

Now flush the privileges to update the database.
mysql# flush privileges;

Finally exit MySQL.
mysql# exit

3.3 Configure Redmine

The database is set up, the next part is now to configure Redmine to make use of it. Navigate to where you have Redmine and enter the directory, which should be /srv/redmine/ if the guide has been followed to the note.
# sudo cp ./config/database.yml.example ./config/database.yml

Now enter in the copied file database.yml and make the following changes to the production section located near the beginning.
# sudo nano ./config/database.yml

Configure the production:
    adapter: mysql
    database: redmine
    host: localhost
    username: redmine
    password: redmine-password

3.4 Generating Redmine Web Server

The Redmine database should be completed, so the next part is to actually make use of it. First the ruby on rails session must be create.
# sudo RAILS_ENV=production rake config/initializers/session_store.rb

Next part is to create the actual database tables.
# sudo RAILS_ENV=production rake db:migrate

Finally to populate the database with the default data.
# sudo RAILS_ENV=production rake redmine:load_default_data

3.5 Redmine Directory Permissions

Redmine should work right now, but first some loose ends need to be tied up. A new user called redmine will be created to handle the Redmine web server exclusively. Using the current user redmine, you want permission of some directories in the root of the Redmine directory.

Make the redmine user with the group access of www-data (for future uses with other server tools).
# sudo useradd -U -G users,www-data redmine

Add a password for the new redmine user.
# sudo passwd redmine

Change the owner of the directories to the redmine user and the www-data group (for future uses with other server tools).
# sudo chown -R redmine:www-data ../redmine/

Change the permissions of the directories.
# sudo chown -R 755 ../redmine/

3.6 Redmine Automation

Finally the next part is to ensure that redmine will start up when you boot up your system. This will be accomplished through the use of the crontab.

Log in as the redmine users first.
# sudo login redmine

Allow the editor to be nano (for simplicity).
# export EDITOR=nano

Now enter the crontab.
# crontab -e

We want Redmine to start when the system reboots so enter this on one-line.
@reboot cd /srv/redmine/; ruby script/server thin -e production

4 Conclusion

Redmine should start up when you restart the system. To visit Redmine you can use your browser and navigate too http://localhost:3000. It is key to note that Redmine will default to port 3000, if you want to change that you can edit the database.yml and add a port field.

The default administrator account for your Redmine web server is admin:admin (account:password).

Introduction

Why this blog?

First of all, I like to spend time fiddling with random settings and configurations with Linux and other software tools. Eventually I get something working out of it (and sometimes nothing to show but wasted hours). The lessons I learn and undocumented details I solve, I never share back to the community. For example, I usually don't reply on Internet forums when I find the solution to someones problem. The primary factor was that I never really put time into it.

I have various conversations with friends on what I am struggling with and what did to solve it. They eventually convinced me to publish the knowledge I gain or just random nuggets of information I discovered. To me I guess that it is fair, karma would agree that I shouldgive back to the community that I have been using and taking for granted for far too long.

Now to finally answer the main question at hand, why? I guess it is to pass on information that might help someone, or that someone might find interesting. I wouldn't expect too much activity out of this blog though, as I am intending to put stuff I find interesting or relevant (which is a niche field I would say). I also do not want to spend too much time maintaining the blog.

What is Coming Soon

Over the next week I plan on releasing a few posts about Software Project Management and Collaboration. Specifically related to the project management tool Redmine and distributed source control system Mercurial.