My Dilemma Of Semantic Versioning

Hello. I am a gem author and I got a problem. It’s called versioning.

Yes, you’re right. _Versioning_. It’s the third big problem of computer science, after expiring caches and naming things.

My gems are all under active development. For some strange reasons, I am avoiding _major_ releases as people might be expecting a _major_ new break-through feature, but all I changed was a _minor_ detail that will affect 1% of all users.

I do minor version bumps instead and try to deprecate as much as possible. Apparently, this is not enough as there’s always angry users with broken builds that “didn’t expect that change with a minor release”.

h3. What Is Semantic Versioning?

The idea of “semantic versioning”: is very clever, and I’m happy there’s something close to a standard.

A semantic version is a string like @1.0.3@, the three segments are called major, minor and patch version. You increment major when you made an “incompatible change”, meaning that the new gem version might break your old code. The minor is bumped when the change is “backward-compatible”, so your code is guaranteed to work with the updated gem. Patch is used when incorporating bug fixes and feature additions.

h3. Semantic Versioning And Gems.

The problem arises when you’re actively working on your gems – which I happen to do! Actively in terms of adding features, fixing bugs _and_ *constantly refactoring* older parts of code. When I refactor stuff I often find better, cleaner ways to achieve something and not uncommonly the restructuring comes with a “minor” behaviour change of the code.

Here are two examples to illustrate my dilemma.

A while ago I started moving common code I use in all my gems into the “uber”: gem. For instance, in representable I allow “dynamic options”.

property :title, if: Rails.env == "production"
property :title, if: lambda { policy.ok? }

That’s a common pattern found in many gems. The @:if@ (or any other option) allows providing a string, a lambda, a symbol, etc and evaluates that at run-time.

Evaluating those options is now handled by uber – which saves me a lot of work. In uber, a dynamic option lambda _always_ receives arguments (at this point it is totally irrelevant what options).

h3. Breaking Things.

So, after updating from representable 1.7.8 to 1.8.0, the above code would break as the old code does _not_ accept arguments.

The fix is to change code as follows.

property :title, if: lambda { |*| policy.ok? }

It’s a really really simple fix. It basically says “I know there might be options passed into my block but I don’t care”.

I didn’t deprecate the former behaviour, but that’s another issue. I knew this was gonna break “some” people’s code. I announced that in the “CHANGES”:, blogged about it, and supported wherever I could.

I didn’t deprecate it as the new solution is way cleaner and consistent – all options receive arguments, are evaluated using the same mechanism, and so on.

Not deprecating this change was stupid. However, deprecating things is pain and not every development team has a horde of willing programmers as in Rails core to work on deprecations.

Most users just updated their @:if@ blocks to receive args and moved on. Some users complained – legitimately – that they were expecting a major bump.

h3. Minor Details.

Another example, also in representable, is when I changed the way inline representers are created. This is a completely internal change.

class SongRepresenter < Representable::Decorator
  property :title
  property :label do
    property :manager

Again, the details here are irrelevant. Look at that block.

property do .. end

That’s what we call an “inline representer”. What happens behind the scenes is a new decorator class gets created for that nested block.

Until today, that new class was inherited from @SongRepresenter@. This is wrong as this might lead to unexpected behaviour in the inline representer. In the new version, that new decorator would be derived from @Representable::Decorator@.

And, right – that change is so _internal_ that 99% of the users won’t even notice that change because nothing changes or breaks for them.

However, 1% of your users, I call them _”power users”_, do amazing stuff with your gem – stuff you didn’t even think of when designing the framework.

Their code might break with that update. They will complain and I have to lower my head in shame and admit that I didn’t properly version my shit.

What makes me frustrated here is that you simply _cannot_ deprecate it – there’s no way to find out if a users relies on the old superclass for the inline representer. It’s a dilemma.

h3. Semantic Versioning – Done Right.

The solution I suggest is to add a fourth segment to the semantic version and shift its meanings down one level. That would mean @ to @ is not backward-compatible. I find this a stupid idea and I know you hate it, too.

That’s why the only way to keep my beloved users happy is to do semantic versioning right. I have to consistently release major versions whenever I change behaviour without deprecating it.

This would lead to fast growing major numbers, e.g. @14.1.0@ or something like that. I am not sure about the acceptance in the Ruby community for that.

To me, it feels strange to see those version numbers. Others, like my friend Ricardo, “find it totally ok”: Ryan “refers to Chrome”: which is currently at v35.

It appears that semantic versioning was designed to have big major numbers – how would you follow an innovative path for development otherwise?

What’s your thought on that? Fast growing majors? Catch-all deprecations? No changes at all???

18 thoughts on “My Dilemma Of Semantic Versioning

  1. Another vote for big version numbers here. Instead of a major number change, mogrify the gem name slightly: “super foo” for example, or change the codename: jaguar to lion, or whatever.


  2. Another option that I’ve seen many gems use is to have a release schedule that allows them to roll up multiple small breaking changes into a larger, periodic releases.

    If people want the cutting edge, they can always point their gemspec at the public repo and grab the latest and greatest changes that way.


  3. Bryan: Isn’t the gem name like a trademark, often associated with a website, a logo, etc? If you change that, it’s like changing your companies URL?

    David: So the periodic rollout means “backward-incompatible” changes are coming without a major bump? Hm, interesting option. Not sure if I could manage a schedule, though, being pretty busy with all my stuff…

    Justin: Thanks – I just said “incompatible, new gem breaks old code” which is the definition of “backward-incompatible”, as you say. My problem is that I know the definitions but I don’t wanna follow strict semantic versioning.


  4. >What makes me frustrated here is that you simply cannot deprecate it – there’s no way to find out if a users relies on the old superclass for the inline representer. It’s a dilemma.

    With static type languages it’s not that horrible because compiler would reveal the problem.

    But the following can be also done in projects where dynamic language is used. When Scala team released a new version, they just collected some open source projects and tried to run tests with a newer version of Scala.


  5. Go for it! Major bumps means improvement. And we’re all wanting it.

    I think I can be classified as a “power user” but I don’t see a problem refactoring whenever I can to keep up to date with a major version bump.

    And while we don’t have time to refactor because we just need to move on, the ~> does the job.


  6. There should be no hang up of old habits or emotions tied to the “major” version number when approaching semantic versioning. Semantic versioning is pragmatic, not emotional or aesthetic. It’s meant to tell you the hard facts, not give some vague indication of:
    “there’s a new major version number, something that should only happen once every few years, so there must for sure be loads of new features and a complete rewrite of the underlying architecture in this one”

    So if you want to break your current api and trouble your users for every new gem version you release, feel free to increase the major version number. That’s how it’s supposed to be done. And that’s just fine (except maybe your users would feel that you ought to think of more stable api’s in the first place).

    About the sperm operator – yes – obviously you shouldn’t use that on major versions since that indicates breaking features. However the minor version can be used with the operator just fine since those changes should be backward-compatible.

    If a gem supports a wide range of even major version numbers however then >= + <= can be used a per usual. Managing gem versioning on open source projects obviously does not get easier when people are not using semantic versioning, as then you can't trust anything about the version numbers and will have to refer to Changelogs etc to figure out if a new version is supported.


  7. +1 for fast growing major numbers.

    I also don’t see a problem with the spermy operator (or ‘pessimistic dependency’ as the rubygems guides call it). gem ‘minitest’, ‘~> 3’ will get me 3.5.0, because it’s just a shortcut for [>= 3.0, < 4.0], which is the latest release in the 3.x series, which should be backwards compatible with 3.4.0. Or maybe I'm missing something?

    Sure, a gem's versions history will grow quickly (f.e., but there's nothing painful about that. But test breakage and fixes after a minor version upgrade on the other hand… A normal rails project has easily 30-40 gems in the gemfile + dependencies. When I update the dependencies (for bugfixes and security) I don't want to digest all these changelogs to see if the author snuck in a breaking change. It also completly breaks any attempts to automate this sensibly. When I upgrade a library to the next major version, I'll expect some breakage (a tradeoff for new features usually) and I'll plan some time for fixing it.


  8. Hakan, Avdi: I see what you mean. Nevertheless, we’re talking about more or less “established” gems like Cells, Reform or Representable. They are definitely “in development” as it terms of I am working on it and I want to innovate and not just maintain.

    There’s no point ever when writing a (complex) gem and you feel like “this is the final API”. There will always be moments of feedback/reflections (“Oh, I didn’t think about that, yet!”) and situations where you realize you did something wrong (“Damn!”).

    Unlike Rails, I explicitly do not want to drag on a design flaw to the next major version. In my opinion, this is contradicting “innovation”. That’s why I am in the dilemma I describe above.


  9. Jeff, David, LHM: The thing is, my APIs are kinda stable, it is minor changes that break 1% users’ code. If I consider them, I have to bump the major version.

    The result will be that 99% of the other users will expect their code to break, or – even worse – expect feature xyz to be implemented.

    The problem is not that my APIs change, the problem is that edge cases are not covered anymore.

    Also, how would you specify compatibility to a fast-growing major version gem? Like ">= 3.0", ">= 4.0", ">= 5.0"?


  10. It’s just a number. In a couple of thousends years every gem version will be at least greater than 1k. But for now, having the confidence, that nothing break, it i run bundle update is a trimendious win. Even if we all had 100% test coverage, it would still be!


  11. “My problem is that I know the definitions but I don’t wanna follow strict semantic versioning.”

    That’s the point of standards. They’re not perfect for everyone, in fact they might be perfect for no one. And if you’re just working by yourself choosing to follow them is optional. But some basic reliable behavior is necessary to coordinate groups of people across space and time.

    In your case you want to: innovate not just maintain (admirable), get positive changes out as fast as possible to your users (good), not have an overwhelming number of versions (nice), not break your users’ builds with misleading versioning (absolutely non-negotiable).

    Your problem is that you can’t have all of these namely: your major versions changes aren’t really big enough to really be called “major” (super minor)

    Just for a second imagine the open source ecosystem if every library, gem, plugin maintainer used the versioning system which fit their use case best. Small, thinly spread, teams become limited to only the biggest projects because devs need to keep track of each project they’re using. Big teams have the resources to keep track of everything, but they’re SOL if they make multiple updates and made a mistake about which project is using which version.

    Semantic versioning is bigger than Cells, it’s bigger that Rails, or Ruby. It’s an industry wide standard because it keeps the machine running. Just follow the standard and be happy that 10’s or 100′ of thousands of people across the globe and decades can agree on something this simple and expressive as this.


  12. You can shift your version points one slot to the right and still be semantic. Just document it. My semantic version pattern is typically `gestalt.major.minor`. The gestalt number represents some major design shift, rewrite, etc.

    Don’t let others stick you with the “one way” crap.


  13. In my opinion, the second example should not be considered a breaking change to the public API, and therefore not require a major version change. It’s unfortunate it causes a break for some power-users, but relying on internal details or undocumented features are at your own risk.
    The first example however, is a breaking change and should have been accompanied with major version bump.


  14. So I don’t normally comment on these kind of articles, but this time I just can’t resist the urge..

    I stopped reading at the line that read “and not uncommonly the restructuring comes with a “minor” behaviour change of the code”.

    Seriously though, either use semantic versioning or don’t. If you can’t update a library without having to make code changes in whichever system consumes the library’s API, you’ve made a breaking change to the library.

    For what it’s worth, I actually work with a software dev team and I handle all the versioning, documentation and release processes for multiple proprietary libraries and application on 3 different platforms.. It’s developers who try use semantic versioning and still break compatibility in “minor” or “patch” updates that make life difficult for the rest of us.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s