I feel like there’s two very different kinds of software that label themselves as “CI servers” and that picking the right one matters.

You can roughly divide most things up based on whether their primary object, the primary entity on which they operate, is a code repository or a more generic “task”/”job”

Code Builders

Think TravisCI, think CircleCI, think Drone.

You point them at a code repository (usually on GitHub), they do things when commits happen; they’ll build the code, run tests, push artifacts, hit webhooks etc.

Config usually lives in a file in the repository being built, because the repository is where it’s all at

Task Orchestrators

Think Jenkins, think Bamboo, think GoCD.

You set up a “task”. A task might be triggered by a commit to a code repository, and cause the code to be cloned and built and deployed to a staging environment somewhere. If that task completes successfully it might kick off two other tasks (fan-out), one doing GUI testing with Selenium, one doing integration testing of an underlying API. If those two tasks complete successfully, they might kick off another task (fan-in) to promote the build to a production environment.

Config usually lives in the service itself, usually edited via the service’s web GUI (unless someone got all fancy and you use something like the Jenkins Job Builder in which case your config exists as code that a Jenkins Job uses to configure other Jenkins Jobs with and it all gets a bit meta!)

Which and why?

As with most things, it’s less a question about what can be done and more about what should be done.

Code Building is essentially a specialisation of Task Orchestrtion, thus it’s easier to use a Task Orchestrator as a Code Builder, i.e. you can just use Bamboo to run your tests. You’ll probably have to do a whole bunch of work to get equivalent functionality though.

Want your PRs to be tested? That means the third-party GHPRB plugin for Jenkins. Want your build/test/deployment config to live in the code repo? At the time of writing it doesn’t look like there’s a good way on Bamboo, though the Jenkins Job Builder is a thing.

Fortunately, you can use both types of tool to get the best of each.

I suspect most people will want to start out with a Code Builder. They have a couple of lumps of code, and what they care most about is “does the code build and do the tests pass”. As they grow things out, and find they need more coordination between multiple jobs, that’s a great time to bring in a Task Orchestrator and start triggering Tasks in response to webhooks from the Code Builder; the best of both worlds.

What makes me sad is seeing people start out by setting up Jenkins, then rolling out their own bespoke setup with third-party plugins and artisanally hand-crafted build configs in the web GUI, just to run their tests and go green or red.

Pick the tool that’s right for you.