Setting Up A Subversion Repository On Linux
Abstract: protected function _getChangedFiles(array $filetypes = array()) {'php5' => array('svn
In this tutorial I'll explain how to setup a subversion repository for PHP Javascript development. I am the lead developer of Group-Office groupware and the owner of Intermesh.
The SVN repository will be used by multiple users using an SSH key to logon to the server. All users will use the svn system user but we'll use the SSH key to identify the user. A pre-commit hook will be used to validate the PHP and Javascript to comply with coding rules.
We used a fresh minimal debian installation to do this.
Install required packages
Install the required software on the minimal Debian server:
$ apt-get install subversion php-codesniffer php5-cli
Setting up the subversion system user and tunnels
We'll create only one system user that will be reading and writing to the SVN repository. We'll use the authorized_keys file of SSH to tunnel the different SSH keys to the svnserve command.
Login as root to the shell and type these commands:
$ adduser svn
$ mkdir /home/svn/.ssh
$ touch /home/svn/.ssh/authorized_keys
$ chown -R svn:svn /home/svn/.ssh
Now edit /home/svn/.ssh/authorized_keys and put in a public SSH key for each user:
command="/usr/bin/svnserve -t -r /usr/local/svn --tunnel-user=repouser1",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty,from="*" ssh-rsa AAAAB3NzaC1yc2EAAAADAQAadsadadAAABAQCrJDzuDALKnKpv9xMaypcTcHHCfhaGuPEoVNyyZ2CrdwCI/GaB04A3YLjaJgdsdsadaB96Lpt5aoxunayx5n0OhRXZZZAioG8CX4Lvmddi/O2fSTU3qfe5iJ4fXUDMzCsadsadsadsadsadFpbTZnbLn6pMjM3KrX+SZlFuBY3zO/WB6i3vcnKpZ1N2wuWcTeOzFm5AMKJGMb4s7rMkA0WlzLZ0xMdzraJK2I43trZf8DUeFeydYRi29Dm2Sr5Jm1uuFt9E3N8VFaIdb7K3IK954lp8ks2ueEXcWLArVFcrAJ15AWS6fzLFQfJeqGaumHjGZTBPnrAg41KHRYXb9Oj65LquNFCrazN3laLIKh [email protected]
Setting up the repository
We'll create a repository 「testrepository」 in 「/usr/local/svn」 with the following commands:
$ mkdir /usr/local/svn
$ svnadmin create /usr/local/svn/testrepository
$ chown svn:svn -R /usr/local/svn
You might want to control permissions in 「/usr/local/svn/testrepository/conf/authz」. Make sure authz-db = authz is enabled in svnserv.conf.
The URL of the repository root will be: 「 svn+ssh://[email protected]/testrepository」.
Setting up the PHP pre-commit hook
We want to test our PHP syntax and coding style when committing so install php-codesniffer:
$ apt-get install php-codesniffer
Create the file /usr/local/svn/CommitTests.php:
<?php
/**
* Subversion pre-commit hook script validating commit log message
*/
/**
* Class for performing various tests in subversion pre-commit hooks
*/
class CommitTests {
/**
* path to php binary
*/
public static $PHP = "/usr/bin/php";
/*
* path to phpcs library
*/
public static $PHPCS = "/usr/bin/phpcs";
/**
* Commit message string
* @var string
*/
protected $_logMessage;
/**
* Commit files list
* @var array
*/
protected $_commitList;
/**
* Changed files list
* @var array
*/
protected $_changedFiles;
/**
* Subversion repository path
* @var string
*/
protected $_repository;
/**
* Transaction number
* @var int
*/
protected $_transaction;
/**
* Class constructor
*
* @param string $repository
* @param string $transaction
* @param array $tests array of test names to run
*/
public function __construct($repository, $transaction, array $tests) {
$this->_repository = $repository;
$this->_transaction = $transaction;
exit($this->_runTests($tests));
}
/**
* Run subversion pre-commit tests
*
* @param array $tests array of test names to run
* @return int result code, 0 == all test passed, other value represents
* number of failed tests
*/
protected function _runTests(array $tests) {
$result = 0;
$messages = '';
foreach ($tests as $k => $v) {
if (is_numeric($k)) {
$test = $v;
$params = array();
} else {
$test = $k;
$params = $v;
if (!is_array($params)) {
throw new Exception('Test arguments should be in an array.');
}
}
$method = "_test$test";
$msg = '';
array_unshift($params, &$msg);
$result +=!call_user_func_array(array($this, $method), $params);
if ($msg) {
$messages .= " *) $msg\n";
}
}
if ($messages) {
$messages = rtrim($messages);
fwrite(STDERR, "----------------\n$messages\n----------------");
}
return $result;
}
/**
* Get commit log message
*
* @return string
*/
protected function _getLogMessage() {
if (null !== $this->_logMessage) {
return $this->_logMessage;
}
$output = null;
$cmd = "svnlook log -t '{$this->_transaction}' '{$this->_repository}'";
exec($cmd, $output);
$this->_logMessage = implode($output);
return $this->_logMessage;
}
/**
* Get content of file from current transaction
*
* @param string $file
* @return string
* @throws Exception
*/
protected function _getFileContent($file) {
$content = '';
$cmd = "svnlook cat -t '{$this->_transaction}' '{$this->_repository}' '$file' 2>&1";
// can't use exec() here because it will strip trailing spaces
$handle = popen($cmd, 'r');
while (!feof($handle)) {
$content .= fread($handle, 1024);
}
$return = pclose($handle);
if (0 != $return) {
throw new Exception($content, $return);
}
return $content;
}
/**
* Get svn properties for file
*
* @param string $file
* @return array
*/
protected function _getFileProps($file) {
$props = array();
$cmd = "svnlook proplist -t '{$this->_transaction}' '{$this->_repository}' '$file'";
$output = null;
exec($cmd, $output);
foreach ($output as $line) {
$propname = trim($line);
$cmd = "svnlook propget -t '{$this->_transaction}' '{$this->_repository}' $propname"
. " '$file'";
$output2 = null;
exec($cmd, $output2);
$propval = trim(implode($output2));
$props[] = "$propname=$propval";
}
return $props;
}
/**
* Get commit files list
*
* @return array filenames are keys and status letters are values
*/
protected function _getCommitList() {
if (null !== $this->_commitList) {
return $this->_commitList;
}
$output = null;
$cmd = "svnlook changed -t '{$this->_transaction}' '{$this->_repository}'";
exec($cmd, $output);
$list = array();
foreach ($output as $item) {
$pos = strpos($item, ' ');
$status = substr($item, 0, $pos);
$file = trim(substr($item, $pos));
$list[$file] = $status;
}
$this->_commitList = $list;
return $this->_commitList;
}
/**
* Get array of modified and added files
*
* @param array $filetypes array of file types used for filtering
* @return array
*/
protected function _getChangedFiles(array $filetypes = array()) {
if (null === $this->_changedFiles) {
$list = $this->_getCommitList();
$files = array();
foreach ($list as $file => $status) {
if ('D' == $status || substr($file, -1) == DIRECTORY_SEPARATOR) {
continue;
}
$files[] = $file;
}
$this->_changedFiles = $files;
}
$files = array();
foreach ($this->_changedFiles as $file) {
$extension = pathinfo($file, PATHINFO_EXTENSION);
$extension = strtolower($extension);
if ($filetypes && !in_array($extension, $filetypes)) {
continue;
}
$files[$file] = $extension;
}
return $files;
}
/**
* trialing comma's in objects or arrays break internet explorer.
*
* @param string $msg
* @return boolean
*/
protected function _testIECommaBug(&$msg) {
$files = $this->_getChangedFiles(array('js'));
foreach ($files as $file => $extension) {
$content = $this->_getFileContent($file);
if (preg_match('/,\s*[}\]]/s', $content)) {
$msg = "You have a trailing , in your javascript file $file. This will cause problems in IE";
return false;
}
}
return true;
}
/**
* Check if log message validates length rules
*
* @param string $msg error messages placeholder
* @param int $minlength minimum length of log message
* @return bool
*/
protected function _testLogMessageLength(&$msg, $minlength = 1) {
$length = strlen(trim($this->_getLogMessage()));
if ($length < $minlength) {
if ($minlength <= 1) {
$msg = "Log message should not be empty. Please specify descriptive log message.";
} else {
$msg = "You log message is too short ($length). It should be at least $minlength"
. " characters long.";
}
return false;
}
return true;
}
/**
* Check if tabs are used as indents instead of spaces
*
* @param string $msg error messages placeholder
* @param array $filetypes array of file types which should be tested
* @return bool
*/
protected function _testTabIndents(&$msg, array $filetypes = array()) {
$result = true;
$files = $this->_getChangedFiles($filetypes);
foreach ($files as $file => $extension) {
$content = $this->_getFileContent($file);
// check if file contains tabs
$m = null;
$tablines = preg_match('/^[ ]{2,}\}/m', $content, $m);
if ($tablines) {
$result = false;
$msg .= "\t[$file] Indents with spaces found\n";
}
}
if (!$result) {
$msg = rtrim($msg);
$msg = "You should use tabs instead of spaces for indents. Following files violate this rule:\n$msg";
}
return $result;
}
/**
* Check if there are trailing spaces in files
*
* @param string $msg error messages placeholder
* @param array $filetypes array of file types which should be tested
* @return bool
*/
protected function _testTrailingSpaces(&$msg, array $filetypes = array()) {
$result = true;
$files = $this->_getChangedFiles($filetypes);
foreach ($files as $file => $extension) {
$content = $this->_getFileContent($file);
// check if file contains trailing spaces
$m = null;
$spacelines = preg_match_all('/\\h$/m', $content, $m);
if ($spacelines) {
$result = false;
$msg .= "\t[$file] Trailing spaces found on $spacelines lines\n";
}
}
if (!$result) {
$msg = rtrim($msg);
$msg = "Trailing spaces are not allowed. Following files violate this rule:\n$msg";
}
return $result;
}
/**
* Check if files have required svn properties set
*
* @param string $msg error messages placeholder
* @param array $proprules svn properties rules
* @return bool
*/
protected function _testSvnProperties(&$msg, array $proprules = array()) {
$result = true;
$files = $this->_getChangedFiles(array_keys($proprules));
foreach ($files as $file => $extension) {
$props = $this->_getFileProps($file);
foreach ($proprules[$extension] as $proprule) {
if (!in_array($proprule, $props)) {
$result = false;
$msg .= "\t[$file]\n";
break;
}
}
}
if (!$result) {
$rules = '';
foreach ($proprules as $filetype => $rule) {
$rule = "*.$filetype = " . implode(',', $rule);
$rules .= "\t$rule\n";
}
$rules = rtrim($rules);
$msg = rtrim($msg);
$msg = "Some files are missing required svn properties.\n"
. "\t= Rules =\n$rules\n"
. "\t= Files violating these rules =\n$msg";
}
return $result;
}
/**
* Check if files violate eol rules
*
* @param string $msg error messages placeholder
* @param array $eolrules
* @return bool
* @throws Exception
*/
protected function _testEol(&$msg, array $eolrules = array()) {
$patterns = array(
'CR' => '(?:\\x0d(?!\\x0a))',
'LF' => '(?:(?<!\\x0d)\\x0a)',
'CRLF' => '(?:\\x0d\\x0a)',
);
$result = true;
$files = $this->_getChangedFiles(array_keys($eolrules));
foreach ($files as $file => $extension) {
$content = $this->_getFileContent($file);
$rules = explode(',', $eolrules[$extension]);
// create reqular expression for checking eol violations
foreach ($rules as $eol) {
$eol = strtoupper(trim($eol));
if (!isset($patterns[$eol])) {
throw new Exception("Unknown EOL: $eol");
}
unset($patterns[$eol]);
}
$m = null;
$pattern = '/' . implode('|', $patterns) . '/';
$badlines = preg_match_all($pattern, $content, $m);
if ($badlines) {
$result = false;
$msg .= "\t[$file] Not allowed EOL characters found on $badlines lines\n";
}
}
if (!$result) {
$msg = rtrim($msg);
$msg = "There is some files violating EOL rules:\n$msg";
}
return $result;
}
/**
* Tests if the committed files pass PHP syntax checking
*
* @param string $msg error messages placeholder
* @param array $filetypes array of file types which should be tested
* @return bool
*/
protected function _testPHPSyntax(&$msg, array $filetypes = array()) {
$result = true;
$files = $this->_getChangedFiles($filetypes);
$tempDir = sys_get_temp_dir();
foreach ($files as $file => $extension) {
$content = $this->_getFileContent($file);
$tempfile = tempnam($tempDir, "stax_");
file_put_contents($tempfile, $content);
$tempfile = realpath($tempfile); //sort out the formatting of the filename
$output = shell_exec(self::$PHP . ' -l "' . $tempfile . '"');
//try to find a parse error text and chop it off
$syntaxErrorMsg = preg_replace("/Errors parsing.*$/", "", $output, -1, $count);
if ($count > 0) { //found errors
$result = false;
$syntaxErrorMsg = str_replace($tempfile, $file, $syntaxErrorMsg); //replace temp filename with real filename
$msg .= "\t[$file] PHP Syntax error in file. Message: $syntaxErrorMsg\n";
}
unlink($tempfile);
}
return $result;
}
/**
* extension -> standard
*/
protected function _testPHPCodeSniffer(&$msg, array $csrules = array()) {
$result = true;
$files = $this->_getChangedFiles(array_keys($csrules));
$tempDir = sys_get_temp_dir();
foreach ($files as $file => $extension) {
$explodedFileName = explode('.', $file);
$length = count($explodedFileName);
if ($length > 2) {
if (in_array($explodedFileName[$length - 1], array('php'))
&& in_array($explodedFileName[$length - 2], array('tpl', 'min'))
) {
continue;
}
}
if (!in_array($extension, array_keys($csrules))) {
continue;
}
$standard = strtolower($csrules[$extension]);
$content = $this->_getFileContent($file);
$tempfile = tempnam($tempDir, "stax_");
file_put_contents($tempfile, $content);
$tempfile = realpath($tempfile);
switch ($standard) {
case 'intermesh':
$output = shell_exec(
self::$PHPCS . ' -n --standard=/usr/share/php/PHP/CodeSniffer/Standards/Intermesh/ruleset.xml '
. $tempfile
);
break;
default:
$output = shell_exec(self::$PHPCS . ' -n --standard=PEAR ' . $tempfile);
}
$count = preg_match_all("/ERROR/", $output, $matches);
if ($count > 1) {
$result = false;
$syntaxErrorMsg = str_replace($tempfile, $file, $output);
$msg .= "\t[$file] PHP CodeSmoffer error in file. Message: $syntaxErrorMsg\n";
}
unlink($tempfile);
}
return $result;
}
}
Create the file:
/usr/local/svn/testrepository/hooks/pre-commit:
#!/usr/bin/php
<?php
require '/usr/local/svn/CommitTests.php';
putenv('PATH=/usr/bin');
new CommitTests($argv[1], $argv[2], array(
'LogMessageLength' => array(3),
'TabIndents' => array(array('php', 'php4', 'php5')),
//'TrailingSpaces' => array(array('php', 'php4', 'php5', 'ini')),
'SvnProperties' => array(array(
'php' => array('svn:keywords=Author Id Revision', 'svn:eol-style=LF'),
'php4' => array('svn:keywords=Author Id Revision', 'svn:eol-style=LF'),
'php5' => array('svn:keywords=Author Id Revision', 'svn:eol-style=LF'),
'html' => array('svn:eol-style=LF'),
'js' => array('svn:eol-style=LF'),
)),
'IECommaBug'=>array(),
// 'PHPCodeSniffer'=>array(array(
// 'php' => '',
// )),
'PHPSyntax'=>array(array('php')),
));
Set the correct file permissions.
$ chown -R svn:svn /usr/local/svn
$ chmod u+x /usr/local/svn/testrepository/hooks/pre-commit
Subversion client configuration
Our pre-commit hook requires certain svn properties to be set on files. We can modify the client to automatically do this. If you're using Linux you can edit 「.subversion/config」 in your home directory and add or uncomment:
enable-auto-props = yes
In the [auto-props] section of the file add:
*.php = svn:keywords=Author Id Revision;svn:eol-style=LF *.js = svn:keywords=Author Id Revision;svn:eol-style=LF
This makes sure all files have Linux style line breaks in them. The Author, Id and Revision tags are useful to add to Document blocks.
Setting the required properties on existing files
When you modify the pre-commit hook on an existing repository you might have files that don't have these properties set yet. You can change the properties on the command line like this:
$ svn propset svn:keywords 「Author Id Revision」 Example.php
$ svn propset svn: eol-style 「LF」 Example.php
With the second command you might encounter an error. The file may have different line breaks or a mix of different line breaks. In that case you can use dos2unix to convert them to linux line breaks:
$ apt-get install dos2unix
$ dos2unix Example.php
Repository structure
I'll describe the way we work with our repository. You may require a different approach but it may help you on your way. In the root we create:
- trunk
- branches
- stable-1.0
- stable-1.1
- tags
- stable-release-1.0.1
- stable-release-1.1.1
trunk is the original line of development where everything started. At some point it's time to release a stable version. Then we create a copy of trunk in the 「branches」 folder. This is what we call the stable branch. The following command would do that:
$ svn cp svn+ssh://[email protected]/testrepository/trunk svn+ssh://[email protected]/testrepository/branches/stable-1.0 -m 「Creating stable 1.0 branch from trunk」
Now we can make some changes required to release a version or perhaps some bug fixes. When we feel the code is ready for a release we can create a 「tag」.
$ svn cp svn+ssh://[email protected]/testrepository/branches/stable-1.0 svn+ssh://[email protected]/testrepository/tags/stable-release-1.0.1 -m 「Creating stable release 1.0.1」
tags will never be modified!
When bug reports come in we'll fix them in the stable-1.0 branch and create a new tag when an update is released.
Merging
The bug fixes in stable-1.0 should make their way back into trunk too. We can do this by merging the branch changes back into the trunk.
Go in to the trunk working copy or checkout a copy:
$ svn co svn+ssh://[email protected]/testrepository/trunk
Then move into the working copy and enter the merge command:
$ svn merge svn+ssh://[email protected]/testrepository/branches/stable-1.0
This will apply all the changes to the trunk. You may have to resolve some conflicts. Eventually you can commit the merge:
$ svn commit -m 'merged revision 1 to 3 from stable-1.0' branch.
End
This should get you started with your own subversion repository. I hope it was helpful!
