Travis CI's new pricing plan threw a wrench in my open source works

I just spent the past 6 hours migrating some of my open source projects from Travis CI to GitHub Actions, and I thought I'd pause for a bit (12 hours into this project, probably 15-20 more to go) to jot down a few thoughts.

I am not one to look a gift horse in the mouth. For almost a decade, Travis CI made it possible for me to build—and maintain, for years—hundreds of open source projects.

I have built projects for Raspberry Pi, PHP, Python, Drupal, Ansible, Kubernetes, macOS, iOS, Android, Docker, Arduino, and more. And almost every single project I built got immediate integration with Travis CI.

Without that testing, and the ability to run tests on a schedule, I would have abandoned most of these projects. But with the testing, I'm able to keep up with build failures induced by bit rot over the years and review PRs more easily.

What went wrong with Travis CI?

From the outset, Travis CI was built to integrate with GitHub repositories and offer free open source CI. At one time it was showered with praise on Hacker News and elsewhere for its culture and ethos.

But as the years passed, other CI systems became much more popular, until late in 2018, GitHub dropped the bombshell that was GitHub Actions, and it seemed like the outlook for Travis CI went to:

K.C. Green - This is fine dog in fire

Now, to shed a little more light on where, precisely, things went from bad to worse, I put together this little timeline of Travis CI's most important moments in relation to open source builds:

Note that we're not yet caught up to the end of the timeline. There are many other open source developers and projects out there still using travis-ci.org, who either haven't found the time or motivation to migrate to travis-ci.com (or elsewhere, ideally), as evidenced by this massive daily backlog of build jobs:

Travis CI backlog of open source builds

Those who, like me, finished migrating to travis-ci.com in the past couple of weeks to get out of that backlog hell are now realizing that we are going to have to beg for extra build minutes after our 1000 'trial plan' build minutes run out:

We will be offering an allotment of OSS minutes that will be reviewed and allocated on a case by case basis. Should you want to apply for these credits please open a request with Travis CI support stating that you’d like to be considered for the OSS allotment. Please include:

  • Your account name and VCS provider How many credits (build minutes) you’d like to request (should your run out of credits again you can repeat the process to request more or discuss a renewable amount)

Sorry, but no thanks. I don't have enough time in my day to send off emails every few [days|weeks|months] requesting extra build credits so I can continue maintaining my open source projects.

Looking in the gift horse's mouth

As I said earlier, I am appreciative of the acceleration in my own growth and career that Travis CI enabled over the past decade. And there so many other services with generous offers for open source maintainers that I appreciate and try to support in any way I can (often financially, as far as I'm able).

I just fear that, like when Google shut down Google Code in 2015, this will have some cascading affects on some of the smaller and less-maintained open source projects some of us rely on, but don't think about.

And I know personally, this whole project is going to soak up around 4 weeks worth of the time I can devote to my free open source work, meaning that's a month out of my open source development time that's just vanished into thin air. Not fun, and not motivating.

I'm lucky to have many supporters on GitHub Sponsors who have helped make my work more sustainable, but most open source maintainers have either a fraction of the support I do, or none at all.

Migrating to GitHub Actions

GitHub Actions logo tagline

Ever since the January 2019 buyout, the writing's been on the wall—especially after so much of the existing engineering staff was laid off. I had already started building newer projects using GitHub Actions for CI instead of Travis CI.

I had planned on slowly migrating over the course of a few years, since many of my older projects still work, and are used, but are not in active development.

But the actions Travis CI took this month—without any prior notice—forced my hand, and now I'm in the middle of probably a 2-to-4-week-long process of moving everything off Travis CI as fast as possible. Getting behind on build failures for a month would lead to a pileup of small issues that will really bog me down.

Anyways, enough of my sob story. I wanted to link to some of the tools I used to migrate, and show how I automated some of the migration, so you can do your own more easily!

The majority of my open source projects are Ansible roles (a large part of the Ansible content I maintain), and to get them migrated, I automated as much as possible. You can see all the details in this GitHub issue: Convert from Travis CI to GitHub Actions.

For each role, I used this Ansible playbook to write new GitHub Actions CI and release workflows using a template. Then I adjusted all the 'cron' values so the Actions won't all run at the same exact time each week.

Then I created github-repo-manager, a simple little project that uses PyGithub to make changes across all or a subset of my repositories. I used it to add a GALAXY_API_KEY secret to all repositories in my account that began with ansible-role-* (for repositories in a personal account, you have to add GitHub Actions secrets to each repo—for organizations, you can create shared organization secrets... much more convenient!).

Finally, I went in and manually migrated the small part of the Travis CI build definition that was custom to each role (e.g. some have a one-dimensional build-matrix, some have a two-dimensional, and they all have a different mix of OSes they build on).

I had already started building many new projects (both Ansible and Kubernetes-related) on top of GitHub Actions, even presenting on the process a few times (e.g. Continuous Ansible testing with Molecule and GitHub Actions). The news earlier this week just accelerated my migration plans for older content from 'when I get time' to 'right now.'

Luckily, GitHub Actions does pretty much everything I could do in Travis CI (so far... I still have 36 more unique repos to go!), and the only real complaint I have is the continue-on-error functionality is no substitute for Travis CI's allowed_failures option.

There's still a bit of work to be done, and I'm trying to prioritize things so I don't leave my most widely-used projects (like Drupal VM) without active CI for too long, but at least this story doesn't have a bad ending.

Comments

Heh, I just noticed in the open source repository migration FAQ it says:

Travis CI will continue to offer a free tier for public or open-source repositories on travis-ci.com and will not be affected by the migration.

Not so much.

I also saw that, but chalked it up to them not going through their FAQs and updating them.
I'm thinking this must have been a knee jerk reaction to their "abuse". As the abuse and these changes all happened within a single month. Or they were waiting for something like this to happen and felt it was time to make even more money.

It will be interesting to see what happens and how many people migrate away, or find their offering applicable for their use cases. $15 for 25k credits is not that bad of an offering, but if your burning through their 10k free credits in a day, 25k credits certainly isn't going to help.

I had to change CI many times.

From jenkins to gitlab...
From travis to github actions...

So I build this small project that helps me to change faster without touch in real CI logic.

https://github.com/rosineygp/mkdkr

I think your problem is bigger than my mine, but this kind of approach can help you in the future.

I appreciate the write up.

Do you think There is an opp for folks to listen to repo changes and volunteer build and test cycles for projects using a rootless container runtime?

I have 10s if not hundreds of spare cores I could gift to an "oss build/test @home" project.

I'm not 100% sure as I have never used Travis-CI but I have been testing my actions with 'jobs..strategy.fail-fast' would this do the same as 'allowed_failures' ?

I also like your idea of splitting out the lint step, so I have taken this idea on board but I have added an additional 'needs: lint' option to the the Molecule job so that it only runs if the Lint job passes.

you've just been screwed over by a corp changing the deal and you're taking refuge in microsoft github? dunno, doesn't seem like a good plan to me, i'd seek firm foss ground with a selfhost option. have you looked at sourcehut? https://sr.ht/

Same thing happened here with all the free credits getting used the morning of Nov 2 and builds just stopped running. I did submit a request for OSS only credits to last through January (estimated time to migrate away, most likely) that same day, and a few days sent another message asking for an estimated time to hear back on the request. It has now been a week, and I have not heard a response to either message... things aren't looking great, but I'll post an update on the total time waited and outcome when I get a response.

Nice article to spread awareness. I am so frustrated.

Nice writeup, thank you.

On a GitHub project for Drupal I own I posted

"So Long TravisCI, and Thanks for All the Fish"
https://github.com/mondrake/drudbal/issues/189

Economics say that someone has to spend the money for the things you use for free, so it's understandable to change idea, but I agree more advanced noticed would have helped, since OSS developers critically lack the resource that woud be needed to switch here - time.

I started my project to self host all of my Heroku apps on a rpi cluster shortly before github actions was released, but I haven't looked back. Using drone.io running on k3s I'm pretty happy with my solution that will pull new pushes to any watched github repo, build, test, and run on any of my domains even .dev thanks to cert-manager.

Nice summary. It feels odd that I didn't see this in "news", since it's showstopper for many projects. Thanks a lot for Actions example, it helped me switching. I'm only afraid that GitHub will announce something similar and we're damned again.
I actually wouldn't mind the credits on travis if it was per month, but one-time budget just doesn't work for me. Also I would go with some kind of "bring your own runner" option, although I understand it wouldn't be easy to implement homogenous environment.

Ah, yes. That would've been a much easier pill to swallow, as it would've at least allowed me to continue using it with my own hardware.

I think Travis CI seriously screwed this up. They screwed it up by offering nearly unlimited time to open source projects almost a decade ago, and inevitably setting themselves up for financial failure. They screwed it up by switching to a new aggressive (and confusing) pricing plan (likely to make up for the losses accrued from the previous decade), leaving likely the vast majority of their user-base without CI. I would be really _very_ surprised if Travis survives much longer after this. But I don't think it's fair to ignore the fact that for 9 years, the _did_ donate their servers to open source projects entirely free of charge. Unlike Google Photos or Google Play Music, this doesn't feel like a marketing play; it feels like a company that was being too altruistic for its own good. And now that they have to pay the price for being too starry-eyed for so long, it is a little unfortunate how few of the many developers--even professional developers!--that benefited from them are now unwilling to even consider paying a small price for those decades of service. Even if not forever, but for a few months to give you the time to migrate to GitHub Actions without all the rush. But I also realize that's not entirely fair either. Everyone sets their own budget, and if you can get something for less, then you can spend your budget on something else, so why wouldn't you? But it was a little frustrating to see that you not once even debated or weighed the pros and cons of paying for it.

And I can't help but see it as ironic that so many people are switching from Travis CI to GitHub Actions. Servers will _always_ have a continuous cost associated with them; there's no way for them not to, so it seems unlikely that a free option will ever be free for a long period of time. So GitHub Actions, like Travis CI, like Google Photos, like Google Play Music, will, after offering itself up almost entirely for free and building a strong following of people who depend on it, or, once it realizes that it can no longer sustain itself, pile-on friction and restrictions to become more profitable. If we disapprove of this practice, then we should try to break the cycle; don't support something that's "suspiciously" free. Self-host a Jenkins server, or find a service you like and that's reasonably priced and pay for it. CircleCI has a free option for open source repos -- _With actual reasonable restrictions!_ That alone makes it seem more stable than GitHub Actions. All GitHub Actions says is "Public Repositories are Free (We love open source)". Mmhmm. So is it the love that's paying for the servers?

But anyways; personally I think I'm going to look into setting up a Jenkins server. It's proper open source, and the GitHub integrations are much more mature than I expected. Although it does have a bit of a DevOps-y learning curve, the work seems net-positive. And being able to keep a Jenkinsfile in my repo makes it feel very Travis-y and easy to configure by others as well.

Great write-up! I honestly couldn't believe it when I saw that the 10,000 credits were non-renewing 😐. My org ran out just a week ago, so I thought "Oh, if I just trim my Travis file a little, next month I should be fine and not hit the limit". *Sigh* so naive. Although if they had just placed reasonable restrictions on time earlier, then I would have streamlined it earlier, they would have had less server utileage, more people would want to pay for the more time, and *sigh*.... anyways. Although I think we have become a little wasteful because Travis was free. Making our CI efficient should be a priority; not just for price but also for the effect that time spent running on a server has _actual_ costs, on e.g. environment, electricity, etc. In all fairness, it is a little surprising that you were able to burn through 10,000 credits in 1 day; that's 1,000 minutes (*~17 hours*!) on a Linux box in a single day! And presumably you needed more than that. That's a pretty substantial amount of computation.

Anyways, anyways; this is turning into an essay 😅. Thank you for the great article, and for all your wonderful open source projects. 🙇‍♂️ I've seen your name pop up in many ansible playbooks, and I now have a face for the name :) Best of luck with your migration!

References:
- https://www.jenkins.io/solutions/github/
- https://circleci.com/open-source/
- https://github.com/features/actions
- https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-m…

unwilling to even consider paying a small price for those decades of service

I would happily pay a 'small price'—for example, I'd be willing to pay $5, $10, maybe even $20 a month for the amount of usage I have. But the lowest price plan is $69/month. Even for a month or two, that's a huge stretch past "free", and even with that, I'd go from 5 to 1 concurrent job, and I would not be able to have my existing CI structure work with that limitation. Moving up from there you have $129/month, and that's just right out.

Docker Hub had a similar kind of issue—and I am happily paying them $5/month now for the privilege of continuing to use their services for almost all OSS images (just one or two tiny private images hosted there). If Travis had a $10/month OSS maintainer plan I would be happily paying it, and might even consider slowing down migration off their platform.

RE: "GitHub Actions ... will, after offering itself up almost entirely for free and building a strong following of people who depend on it, or, once it realizes that it can no longer sustain itself ..."

GitHub is owned by Microsoft, who has about a billion cloud servers sitting around making money selling processing time to the world. I think they can absorb a few thousand free open source hours.

I think the main problem with Travis was that it was first. Everyone else was observing the mistakes they made and was designing something better. I think the real breakthroughs were the step-based approach of Gitlab CI (and Circle CI, I think) and the actions of Azure DevOps that would become GitHub Actions. Travis tried to introduce some of these (stages), but the offering from the newer contenders was just leagues better in providing simple, composable, maintainable, readable CI scripts that worked on all OSs and could control dependencies, rather than looking like Linux bash scripts that break on every OS update. And they were explicitly designed to handle multiple workflows, rather than having one workflow built only for testing (and hacked for deployment later). And when you look for .travis.yml examples, or existing CI, you see the old format. The fact that MS could provide 10x the runners vs. Travis was also huge, of course; but the real reason for me is the fact I can easily read and maintain dozens of repos. And dependabot was a brilliant addition to GHA, as well, since it allows them to automatically make PRs to update actions or suggest things if there are security updates that need to be made. They still don't have `allow_failures`, `[skip ci]`, rerun failing jobs, rerun selected job, or other things.

Circle CI also has a limited OSS model, but they have done much better than Travis because they had a better design. Again, mostly better because they were not first. It is very sad to see them going, though, as Travis has contributed a lot to OSS over the years.

Hi Jeff,

thank you for this post (and for the support you gives to Open Source community). I've just discovered that Travis CI has setted up the Partner Queue Solution (https://docs.travis-ci.com/user/billing-overview/#partner-queue-solution) which enable execution on sponsored infrastructures at no cost (for more informations, see: https://community.arm.com/developer/tools-software/tools/b/tools-softwa…).

Briefly, you can keep using Travis CI without consuming credits but the .travis.yml file need to change a little in order to enable this. However the sponsored architectures are on arm64 VM and on LXC containers in both ppc64le and s390x which are quite limited: for example if you rely on docker and docker compose, you have to take this into consideration while developing and when using Travis CI, for example by building docker containers for arm64 architectures. At the moment The s390x seems to be the most similar architecture to amd64, and I plan to test my project relying on this configuration

From what I understand, many CI providers are having to change their free tiers because of extensive abuse from crypto miners on their platforms.