Make Your Specs Faster with Poltergeist. And Fail.

Some time ago we decided to make our acceptance tests faster. We were using Cucumber with Selenium and we replaced it with the Poltergeist driver. Poltergeist uses the PhantomJS engine and thanks to that our tests run around three times faster then they did before. Everything works smoothly on our machines, but there is one small problem; Sometimes, in some steps PhantomJS crashes on CircleCI :).

This forces us to click “rebuild” a few times in a row. This doesn’t make our tests faster, but the direction is good. So what could we do? We could:

Rerunning Cucumber Scenarios illustration

  1. Connect with CircleCI VM using SSH.
  2. Download the crash dump.
  3. Notice that it contains sensitive data.
  4. Report the crash by creating an issue on GitHub. (I surrender)
  5. Wait for someone to fix it or fix it by ourselves.
  6. Wait for a new version of Poltergeist.
  7. Wait for CircleCI to update their Poltergeist version.

Or maybe…

Rerun Failing Specs

Cucumber, as with most of the testing tools out there, allows you to choose an output format. What’s more, is it has one specific format called rerun which writes a list of failing scenarios to a specified file.

cucumber -f rerun –out failing_scenarios.txt

Once you have this file, you can run these scenarios again:

cucumber @failing_scenarios.txt

It’s as easy as that! Let’s write rake tasks which do this:

cucumber logo At the beginning I was afraid that this would not work with parallel nodes; failing_scenarios.txt shouldn’t be shared between them. But every CircleCI node is an independent virtual machine, with it’s own filesystem, so every node has separate file.

Now you can type rake failing_cucumber_specs:record and rake failing_cucumber_specs:rerun.

I also updated the test section of circle.yml:

It’s good idea to put failing_scenarios.txt to the .gitignore file before committing changes.

Usage with Knapsack

We use Knapsack (written by Artur Trzop) which splits tests among multiple nodes. Knapsack has it’s own adapter for Cucumber, so I had to modify the failing_cucumber_specs:record task. Here is a version for Knapsack:

knapsack logo

Possible Troubles

Exit 0 Is Not a Perfect Solution

If you look closely at the rerun task, you can see exit 0 after running Cucumber. We must return a successful exit code, because we don’t want our build to be interrupted during recording failing scenarios. The problem with Cucumber is that it returns 1 when some scenarios fail as well as when it fails itself for any reason. Imagine such a situation:

  1. Cucumber doesn’t run specs, creates an empty failing scenarios file and crashes.
  2. CircleCI doesn’t notice that, because we force exit 0.
  3. Second Cucumber execution run specs from an empty file. No specs, so it returns 0.
  4. Build is green.

Fortunately, the first point seems to be very unlikely. Even if Cucumber fails for any reason other than red specs (that’s already unlikely), it doesn’t create an empty file, so the second Cucumber run fails. However there was a feature request regarding Cucumber exit status codes. It’s implemented and merged to the master branch so in future releases we will be able to determine whether scenarios failed (exit status 1) or application returned error (exit status 2).

Less Trust in Specs

Imagine some functionality which doesn’t work as expected from time to time, let’s say, because of a race condition. This problem could be noticed when it’s test fail. Rerunning failing tests decreases the probability of detecting this issue. I don’t think it’s a huge problem in our case as I’ve never encountered this in any project I was working on at our company, but I feel obligated to mention this.

Open Salaries: When

Paweł Brodziński May 5, 2015

Once I started playing with the idea of making salaries at Lunar Logic transparent, the question that popped up almost instantly was: when? Well, one thing that I should probably start with was to ask whether we want to do that at all.

If I simply ask the latter out of the blue I would likely get mostly negative feedback. This is a huge change and one that may make people uncomfortable.

The answer for “when” question would thus likely be “whenever we are ready.”

I started with sharing with everyone that there is an idea to go transparent with salaries at some point in the future. I got some early feedback on that, including concerns on what would happen once we have open salaries.

This gave me confirmation that transparency alone is not enough and there has to be an accompanying mechanism that allows people to influence how salaries are set.

Then there was a question of how fair we were with salaries at that moment. Well, from my perspective it was fairly fine of course. However, the moment of making salaries open to everyone would mean that we are suddenly taking into consideration everyone’s opinion, and not only mine.

I asked a few people at the company to prepare their abstract salary list. They could use whatever reference point they wanted, either they own salary or just a completely abstract number, like 100. Then they were supposed to relatively set other employees on the scale. I wasn’t very strict with that. “No opinion” was a perfect option, as well as an incomplete list or partial information, if someone decided to use a whole set of factors.

I gave that task to a few people in different roles and of different characters to get a possibly broad set of data. They were working individually.

The outcome was some sort of an abstract, and partial, verification how my views on salaries differ from opinions of others.

One result of that was my further work on flattening the salary scale – a process that was in place for some time already. There were a couple of cases when I realized that someone should have gotten a raise already and fixed that too.

Childern in gnome hats taking a close look at gems and coins

Concurrently more and more informal chats around the idea were happening. Given a well-thought approach to the process, more and more people were buying into the idea. At some point, I felt that majority of us were supporting the idea.

The last missing bit was figuring out how the change to transparent salaries would happen and what will be the mechanism of influencing salaries from that point on. On one hand, this part wasn’t easy. On the other, stories from companies that are already doing that are available. A few examples that come to my mind would be: Buffer, Semco, and a few case studies covered in Reinventing Organizations.

I used these stories more as an inspiration than a recipe. Eventually, I ended up with an idea that was ready to put under the scrutiny of the whole team.

We were ready.

By the way, if you are interested, the whole preparation process took 9-10 months.

Open Salaries: Why

Paweł Brodziński April 28, 2015

Transparent salaries are becoming increasingly popular. I know more and more companies that decide to change the traditional approach and make salaries known within the organization. Buffer goes as far as publishing their salaries to the world on their blog.

We did the same at Lunar Logic.

Pursuing open salaries just for the sake of doing it doesn’t make sense, though. What were our motivations to go down that path?

Boy in a gnome hat not knowing what to do with all the gems and coins

Decentralizing control requires decentralizing both the authority to make decisions and the information required to make these decisions correctly.

Don Reinertsen

First of all, we are evolving toward no management. This means that we want everyone to be involved in everyday management. Now, for a service company, such as Lunar, more than 80% of all our costs are labor-related. Without information about salaries, involvement in leading the organization simply can’t go too far.

Collective intelligence was much more predictive in terms of succeeding in complex tasks than average individual intelligence or maximal individual intelligence.

Anita Woolley

Another reason is fairness. The rule I’ve followed for years, whenever discussing salaries, is that I want them to be fair. Fair for an employee but more importantly fair for the whole team, group, organization. The problem is that it was me who was deciding what’s fair and what’s not.

Individual intelligence won’t beat collective intelligence on that account. I wanted to get more people involved in the process as this would let us make better, fairer decisions.

Children in gnome hats deciding what to do with the gems and coins

Finally, I realized one thing when looking for stories of companies that either had open salaries from the beginning or changed to such model at some point. Transparent salaries, once in place aren’t much of a problem.

What happens to be problematic is the moment of the change. This, however, may be prepared for and managed so that the negative impact is not that strong.

There is one more thing that is a consequence of arguments pointed above. I do assume that a single person acting in best faith won’t be as fair as a group can be. This means that despite the fact that I chased fairness I must assume our salary list unfair to some degree.

The outcome of this is that sharing information about salaries alone would trigger frustration. Some people would consider some salaries unfair and, unless they could do something about the situation, the only reasonable outcome would be frustration. This means that it’s not only about information but also about control.

Transparent salaries must go along with a mechanism that allows everyone to influence how salaries are set. As Don Reinertsen points: it is information and authority.

That is, in short, why we decided to make salaries transparent and at the same time introduce a mechanism that allows everyone to influence what everyone’s salary will be in the future.

Playing with black matter – continuation

Hanna Seweryn March 24, 2015

pg_morphSome time ago I started experimenting with PostgreSQL features to allow creating foreign keys for ActiveRecord polymorphic relations. Since PostgreSQL is a pretty powerful tool, I ended up creating a pg_morph gem which made it much easier to achieve. How it works I already described in my previous post, but if you don’t want to go back in the past, I’ll give a short refresher.

Pg_morph takes advantage of inheritance and partitioning features of PostgreSQL. For a main table which contains records associated with different tables it creates partitions in which proper records may be stored based on their association type. All those partitions inherit from main table, so by running searches on the main table you can retrieve data from its partitions. It has one additional advantage, which is searching by only one type of association requires looking into only one partition, the others are omitted, so queries may be notably faster.

From the ActiveRecord’s point of view nothing has changed and this was also a goal for a new version of pg_morph, which took magic even further beyond ActiveRecord.

So what has changed?

The initial version had one caveat, but a serious one. Because all inserts to the main table had to be redirected to partitions, there was nothing created in the main table and RETURNING id statement at the end of each INSERT wasn’t returning what every Ruby on Rails programmer is used to – newly created records had id equal nil. Not nice. To bypass this behavior, PostgreSQL was allowed to create records in the main table, and then, after insert, the duplicated record was removed.

To avoid such ugly solution I added new layer on top of all tables, which is the view of the main table. Having that, there is no need to overload database by creating redundant rows and deleting them immediately after. Thanks to that, additional operations which may cause some problems, are avoided. The main table remains empty so whole structure of partitions inheriting from main the table is build in accordance with PostgreSQL specification. And what’s most important is that ActiveRecord now doesn’t have any problems with missing ids of newly created records.

‘Whoa, wait!’ you can exclaim. ‘How can AR know that now there is a view which has to be used instead of a good old table. Does it mean that I should change all my associations to use new relation?’ No, don’t worry, pg_morph takes care of it. It creates view in place of the main table and renames main table once again taking care of possible naming conflicts.

Give me some code!

Let’s take some example data model:

So after running:

you will have a view named images which knows exactly which tables to ask for the records. And what’s most important, both partitions – for users images and items images – will have foreign keys.

Also removing this whole additional structure is as easy as adding that line to your migrations:

It doesn’t matter if you want to remove all partitions or only one particular. It will check if there are other partitions of the main table (in this example case images) which still have to be supported, and in the case that there are no more of them, the view is removed and the main table recovers its previous name. Everything starts to work like nothing has even happened.

Of course there is still room for improvements, so if you see some ways of making it better or spot any problems, simply share. Or even better – fork, and pull request!

Product Owner Internship at Lunar

Paweł Brodziński March 5, 2015

Signpost pointing to Awesome Internship and Lunar Logic Let me start with the remark that I hate the title of this post. The product ownership that is typically understood doesn’t exactly explain what role we are thinking of, yet I couldn’t come up with a better name. Anyway, bear with me, I’ll explain everything.

We are looking for an intern to do a little bit of work around our products and learn like hell for the rest of the time. The person in that role will be involved in shaping development of one of our products. It means a little bit of what’s typically covered by Product Owners and a lot of what is happening within the Lean Startup movement.

It’s not going to be about prioritizing features or even choosing the ones that will get built. It’s about figuring out what the product going to be. There’s nothing written in the stone. Experimenting is the keyword here.

If you are passionate about software but don’t really feel like development is your thing, then this may be a perfect choice for you.

We don’t expect you have an extensive knowledge about the stuff mentioned above. We do expect that you learn like there’s no tomorrow. This is the point of the internship – to find someone who is capable of building expertise in such a role.

Here are some questions that you may want to ask.

Q: What are the requirements?

A: Besides of learning capabilities and passion, English fluency is the only requirement. A lot of sources you’d use are available only in English.

Q: What will I be doing during the internship?

A: Read. A lot. And then some more. In fact, that’s what you will likely do for majority of your time here. You will use what you learned working on one of our products. You will design and run product experiments. You will get out of the building. You will get out of your comfort zone. You will turn an idea of a product into a real thing.

Q: Wait, what? That sounds serious. Is that the role for an intern? My ideas would be ignored by more senior people, I would guess.

A: The whole idea of the internship is bringing specific knowledge to the team and you are going to be the very person who adds that to the mix. Besides we don’t really have a hierarchy at Lunar so no one looks down on interns.

Q: There is someone doing this thing at the company, so they’d mentor me, right?

A: No, not really. That’s the whole point of “learns like hell” part of the requirements. For some, this thing is super fun for others, a struggle. We are looking for the former.

Q: This whole thing sounds crazy but I’m willing to give a shot. How long is it going to last?

A: A half of a year sounds like a good plan but it’s not written in the stone. If you have other ideas let us know.

Q: Whoa, that’s a lot of time. Do I get paid?

A: Yup. This is not a plot to get cheap labor, but rather, to run an organizational experiment. We are serious about experimenting.

Q: So what happens after the internship?

A: If it works out well we will most likely change it into a permanent role. Oh, and if it works out well, we will also have a perfect candidate for the role. You. That would be quite a nice coincidence, wouldn’t it?

Q: How is the hiring process is going to look?

A: First, we want to have a meaningful chat about the ideas of Lean Startup. I mean, we really want you to first read the book and, only then, reach out to us. Other Lean Startup sources can help too but, come on, we already ask you to read the book up front. The goal of that part is for you to figure out whether that’s something you’d like to do and for us to get a hint how your mind is wired and how you learn.

Then, we’d run a demo day with a few people who impressed us most. A demo day is a rather unstructured day spent on site with us where we try to figure out whether a new person is a good fit for us. You may expect some hands on work, a little bit of more formal, and a lot of less formal conversations. It seems that you can expect quite some fun too.

Q: One thing still bothers me. You want to invest so much time to the internship instead of hiring an expert. Why?

A: It’s an experiment. That’s one. Besides, the kind of expertise we look for is, unfortunately, really rare. That’s two. Finally, we care a lot about organizational culture and cultural fit that generally makes hiring a challenge for us. This means that it’s not only about skills but also about how you fit the company. That’s three.

Besides we’ve been through a number of cases of people growing at Lunar and most definitely wouldn’t mind another one.

Q: How to apply?

A: Once you”re ready to talk about Lean Startup write an email to Pawel.

Q: I have another question.

A: Send Pawel an email.

UPDATE: We have finished hiring process for the internships. Thank you for participation.