Developers need some way of ensuring that their changes don't break things. Improved unit test coverage is one thing, but unless the tests are actually run they are not very useful. CI would ensure that tests are run for any new code checked into the codebase.
Travis CI is a perfect candidate for open source projects.
This is already done btw... PR coming soon
We actually set up something a couple of months back using PHPCI (https://www.phptesting.org/) but we never really ended up using it for whatever reason ...
I'm really looking forward to see what you did with TravisCI but I think the fundamental problem here is our (very) poor test coverage.
Stefan – I agree that the basic problem is lack of tests, and obviously this (or any CI setup) won't fix that, but it will encourage others to add tests as they will see that they are actually being run, and of course if there are any changes that break the tests that are there, that will become obvious much more easily / quickly. Plus it reduces the impact of developers "forgetting" (lol) to run unit tests locally before commits, because there is an automated fallback.
Due to the fact that some of the tests require an installed database, I needed to add an "install" command to caUtils, which is also part of the above pull request. I also had to fix a small issue with caUtils itself trying to read the locale out of the database, which works whenever the database is installed of course, but not before you run the "install" command.
Another part of this was the ability to specify a different setup.php for caUtils, so it is now possible to setup a second database for tests and that won't impact on your existing database: `caUtils [command] --setup=path/to/setup.php` Note that these changes are not part of caTools, only caUtils. I've not used caTools so unsure whether it is even applicable
I also had to change the URL for the (only) submodule in use, from an ssh URL to https URL. The Travis instance doesn't have ssh keys, and while they could be installed as part of the .travis.yml "script", I think switching to https makes more sense in this case. It also doesn't hurt in other cases (e.g. actual installations) either, because the submodule is in a public repo and you generally don't want to write back to it when using it as a submodule.
One other thing that has come out of this, although not specifically related to tests, is that the way setup.php-dist is currently being used – copy and modify value(s) as required – is not necessarily the only way to do it. If you look at .travis.setup.php, it sets the values it needs to override, then require()s setup.php-dist which fills in the blanks. (All the `define(X)` calls in setup.php-dist are now protected by an `if (!defined(X))` check). This style could also be used for "real" setup.php files; I think we (WA Museum) will be changing ours to this style. The main advantage is that, after doing an update, you get any new stuff in setup.php-dist, without having to merge it into your setup.php.
P.S. You will need to setup the service integration between travis-ci.org github. It's just a matter of creating an account, giving it access to your github account, and enabling builds for the providence repo. We have it enabled for our origins and the wamuseum fork, but it makes more sense for someone from CA / Whirl-i-gig to control the upstream CI
Well you can only set up repositories that are on your own GitHub account so Seth is going to have to do this