Baruco 2014 Reaction

Phillip Boksz September 24, 2014

And it begins

Baruco 2014 has come and gone in a flash. Like the saying goes, “Time flies when you are having fun”. And so it was; The talks, the introductions, the discussions, and the lunches were all fantastic.

Baruco stage

First day of talks

The first day of talks started off with the creator of Ruby himself: Yukihiro “Matz” Matsumoto. He discussed a new version of Ruby called mRuby that is specifically designed for embedded systems. He went into details about how he managed to get a grant from the Japanese government to work on this project. He expressed his desire to get Ruby code into realms other than web development, specifically embedded systems. mRuby will have Ruby version 1.9 compatibility, but will have a less functionality than MRI (which he says should be called CRuby, since he doesn’t even actively develop for it anymore).

Next was a talk by Piotr Szotkowski, which emphasized the fact that Ruby has a lot of very useful functionality built into its standard libraries. He went through a few examples of using some of this functionality by showing their uses in a gem he created called signore. The first thing he showed was a module called abbrev which is called on an array of strings or symbols and returns a hash that maps each possible abbreviation to the string it matches to. This is something that can be useful to create a tab complete in Ruby. He then showed how using the standard library you can create a simple HTTP server using WEBrick. And a simple TCP server using the gserver module. He also talked about the prime module which simply checks if a method is prime. He wrote his own method, and segued into the benchmark module by showing that the prime method in the Ruby standard library actually took 3 times longer to run than his method. He showed the ease of use of the benchmark module which would later be used extensively in another talk.

Next, Pat Shaughnessy took us into the world of ActiveRecord, to pick it apart and find out how it converts and everyday query like User.where(email: “<email@email.com>”).first into SQL statements, and then how SQL works in order to find this specific record. He showed how SQL will normally do a lot of work by doing a sequential search to find all records that match that criteria then chop off the first one and return it. He showed how adding indexes to things that are commonly searched on (in the above case it would be the user’s email) to speed up the SQL query. With an index it does a binary search through the records instead. He also showed how when there were a lot of records SQL created pagination so that it could work with blocks of records in memory rather than from the disk.

Leon Gersing on stage at Baruco
The other site of programming – Leon Gersing (Ruby Buddha).

Later on in the day after a fantastic lunch (a full 4 course meal: salad, paella, fried calamari, wine, and dessert) Leon Gersing AKA Ruby Buddha, gave a talk that was pure motivation. His point was to step back and view problems in a different light. I have found that that has always helped me. Sometimes a good nights sleep, or a game of Dominion, or even just a quick coffee break can clear your mind, and let you rethink something complicated.

The last talk of day one was by Ryan Levick, who proclaimed his love for Ruby on multiple occasions during his talk. He used the concept of static typing, which Ruby does not have to make one simple point: Ruby is not the right tool for the job every time. There are other languages out there, and even though Ruby is amazing, it is not always the language you want to, or even need to use.

Second day of talks

The next day, we jumped right into talks at 9:00. The second talk of the day was one about writing fast Ruby. Erik Michaels-Ober talked about some subtle differences in Ruby syntax that can improve performance in your app. His goals were simple in this presentation: optimize at code level, improve performance by at least 12%, and maintain high quality, readable Ruby code. Here is a quick summary of what syntax is faster:

  • yield is 5x faster than block.call
  • (1..100).map(&:to\_s) is 20% faster than (1..100).map { |i| i.to\_s }
  • flat_map is 4.5x faster than something.map.flatten(1)
  • hash.merge!(e => e) is 3x faster than hash.merge(e => e) This is assuming that having this hash be mutable is safe
  • hash[e] = e is also 2x faster than hash.merge!(e => e)
  • hash.fetch(:something) { :default } is 2x faster than hash.fetch(:something, :default) – this is because the block is only evaluated if the fetch doesn’t find anything
  • sub is is 50% faster than gsub
    • Most of the time we only want to sub the first thing anyway!
    • gsub evaluates the whole string (global sub)
  • tr is 5x faster than gsub – so why does everyone use gsub?
  • Using exceptions for control flow is 10x SLOWER than a if statement!
  • A while loop is 80% faster than an each_with_index loop

After this Jason R Clark took us on a tour of very useful Ruby debugging tools. He started with the stuff already built in. First up being the infamous puts which, in the grand scheme of things can have its uses. He then showed how the caller method works, which returns a stack trace where it is called (super useful). He also showed how the $! variable works. Whenever it is called in the code it returns the exception that is currently being handled. In the Ruby community we use a lot of gems, and sometime our errors stem from them. So debugging a gem could potentially be very time saving. Jason showed that you can open up any gem files on your current Ruby version very easily by writing gem open <gemname> or bundle open <gemname> in the command line. You can then make changes to the code to help debug. After that you can run gem pristine <gemname> to return the gem to its original state, basically wiping all the changes you made. This will make sure you don’t accidentally commit any debugging changes to your gems. He also spoke about a few useful gems, one of them being Pry. Pry is a gem that you can use to create a kind of breakpoint in the code that then opens an interactive console at that point. You can check instance variables, write code, and move around in the file using unix-like commands (ls, cd, etc). There is also a bunch of useful pry addons to step forward and back in the code so you can see in more detail what is going on.

The final few talks of the day went over various topics, from security to monads, and ended with robots. And dancing. With them. The last speaker had more of a demo than a talk. Julian Cheal showed some of the interesting things you can do with Ruby and robots. He showed the some things you can do with Ruby code to control everything from LED lights to drones. He even set up two flying drones to be controlled by a DDR dance mat and his phone. It ended up being a very spectacular way to finish off the conference.

Playing foosball with guys from Codegram
Codegram should get a gold medal for the great conference organization as well as for the good foosball game. PS. We have the whole year for the training and we will beat you at #Baruco2015 :)

And it ends

All in all the conference was fantastically organized, a pleasure to be at, and a great learning experience. The speakers were all great, informative, and motivational. The lessons were practical, and easily applicable. The problem I have with most talks at conferences is that the ideas are good but they are sometimes difficult to incorporate in your everyday work. This was not the case with a lot of the stuff this year. I am very happy about spending some time in Barcelona and attending this conference. I would like to thank everyone that made it possible, namely Codegram for putting in so much effort to organize such an amazing conference!

Yummy Baruco!

Tomek Rusilko September 17, 2014

Baruco 2014 is over. I was lucky to be there and soak in all that it offered. Let me share some of my thoughts.

Tapas

I arrived to Barcelona early enough to take part in one of the workshops prepared by organizers as a kind of a side dish to the main event. “Test drive a browser game with angular” held by Test Double crew – Zach Briggs and Todd Kaufman – was my choice. I figured this would come useful for my day-to-day work, as angular is our first front end choice here at Lunar. Although in technical terms I did not learn anything new about the framework – the code written during workshop didn’t go past simple controller, few ng-models and some ng-repeat here and there – there was a great deal of added value in what Zach and Todd shared between the lines. They took a stand in the ongoing debate on vitality of TDD, and their approach seems to be moderate yet reasonable – test drive your business logic, test drive complex parts but don’t be dogmatic about it. There are uses and misuses. And it’s crucial to remember that TDD is not only about feeling safe and having your back covered but more importantly is about design emerging from your code driven by tests. Having this in mind Zach led the workshop during which we wrote a pure JavaScript app entirely separated from angular, test driven with jasmine. Tasty!

First course

Tomek Rusiłko's Baruco conference passThe main event started with a blow – dry ice, gig-like atmosphere, and all the conference heroes presented in a stunning animated intro. To be honest – I had mixed feelings about Baruco’s marketing theme. Superheroes? Seriously? You can hardly find a more exploited topic in programming world. But the Codegram crew had really pulled this one off. It was the perfect delivery with all the little details that made it work. And you could easily tell how it boosted everyone’s morale – the speakers’ and the audience. Being part of this made me feel great. But this is ruby community – that’s probably the right way to feel, isn’t it?

On the first day we had a pleasure to hear Yukihiro Matsumoto speaking about his current undertaking – mRuby – the smaller version of ruby interpreter dedicated for embedded systems. He made a nice run through various ruby implementations, dismissing each one of them as not good enough for embedded systems. CRuby* for instance is too much POSIX based. It was interesting to hear that with mRuby it may be possible to write software for vending machines, home automation or even some simplistic versions of satellites. Being a language designer as Matz called himself (as opposed to a programmer) seems like a cool job.

If you launch rockets I hope you don’t use ruby

Next came Piotr Szotkowski, whom I remembered for a great talk about Bogus he gave at this year’s wroc_love.rb conference. I was expecting yet another tasty technical presentation and I was not disappointed. Piotr talked about powerful, but somewhat hidden features of ruby stdlib. He showed good code examples and gave sound pieces of advice.

MIL**: Study enumerable module. Then study it again.

Don’t return nil from your methods.

Warmed up by @chastell, we were ready for heavy artillery. Pat Shaugnessy entered the stage digging deep into ActiveRecord, Arel and Postgres internals. He went through stuff like AST, visitor pattern, yak parser, btree algorithm, db indexes implementations and alike. Demanding topics, yet Pat managed to keep audience interested. Be sure to check out his newest book Ruby under microscope.

MIL: Learn the stuff down there, know the internals of the tools you use at least to some level – it will make you a better engineer.

The only lady among the Avengers – Emily Stolfo – had a much less technical talk on responsible release process.

Emily cleverly laid out her ideas dividing them into 3 main areas: maintaining simple API, clear communication and semantic versioning.

Check out my notes to see more details or watch Emily’s talk if your are responsible for any gems or libraries. I will definitely come back to it when the time comes for me.

MIL: Think of your API as a user interface and provide good user experience to establish and maintain what’s most important – trust.

Speaking of responsibility, enter Jose Albornoz with his talk about … irresponsibility. Coincidence? I don’t think so.

The youngest of the speakers took us on a trip through his personal experience of, what he called, a conference driven development.

In an entertaining way he described how he created the first ruby Gameboy emulator from scratch just to find out that it was… not the first.

Technical details were a bit dry but the main point was clear – find time to write irresponsible code just for the sake of it, learn and embrace the fun of coding.

MIL: You don’t have to know the difference between bits and bytes to write code.

So we are sitting there, half way through the day one and then comes this guy – Leon Gersing – starting with this quote from Las Vegas Parano. Yeah. This is going to be good. It’s impossible to sum it up – just watch it when it comes out, it’s worth your time.

MIL: Thought can’t be replaced by a process. You can’t adopt the culture without understanding it – this goes especially for Agile philosophy.

Also: The Perfect High

Main course

The last talk of the first day by energetic Rayn Levick, corresponded well with the first talk of the second day delivered by Brian Shirai. Both gentlemen talked about a recently hot topic – static typing and type safety, and whether we need it in ruby. Interestingly enough Rayn and Brain found themselves on the opposite sides (mark, not ends) of the problem’s spectrum, former pointing lack of types as a ruby’s weakness, the latter explaining how they can do more harm than good. If I was to judge – Brian presented a much stronger case. His talk was thorough, contained many cross references and left me with a huge material for further study. Having it difficult to grasp all on the spot, I will be definitely coming back to it.

MIL: Types don’t fit where there is much interoperability (objects). Programming is a behavioural science. Proposition as types = logic as types and it is not a good idea because we can’t really use logic well.

Having such a good start into the second day, we were about to see even better stuff. Erik Michaels-Ober enchanted us with beautifully illustrated and content heavy story of how optimizing ruby code for performance can be fun and not code obscuring. It turns out you can write fast and pretty code at the same time. Lots of good tips, simple code examples, each one backed by benchmark analysis. Watch the slides or check my notes for details. Fully professional talk from every standpoint. 10/10. If I had to choose only one of the talks to watch – this would be the one.

MIL: Performance optimizatoin can be fun and even have a therapeutic effect.

Next up was Jason Clark on various debugging practices and tools. Well prepared talk, spanning across the entire scope of debugging options out there. I will be definitely reaching to this presentation in times of need.

There couldn’t be a ruby event without at least one talk about services. Evan Phoenix filled this spot nicely balancing out pros and cons of services, leaning towards conclusion: use services, when the problem becomes too complex to grasp. Definitely worth watching if you never used SOA or felt like you used it wrong. What I liked most is that Evan pinpointed some easy to follow good practices, e.g.: never share AR model between services, start with services that map to boxes drawn on the white board, do a fire drill once a month, have one convention of how services should talk to each other – set it and don’t discuss it too much, just like you don’t discuss method calls.

MIL: Every sufficiently complex problem domain will require an app larger than human cognition mass threshold, and team’s threshold is smaller than a single developer’s.

If an app is never deployed, was it written?

Did you hear about the Monad tutorial fallacy? In short it states that once you understand what a monad is you are unable to explain it to others. Seems like this no longer applies thanks to Tom Stuart (@tomstuart), whose presentation on Monads implementation in ruby left me in awe. Unfortunately I can’t explain it to you. Watch this talk when available, I dare you.

Book tip: Understanding Computation.

At this point my brain was swollen and we had still 3 more talks to go. I didn’t take much notes from them so let me just really quickly summarize: Matt Amonetti told a colorful story about pitfalls of cookie based authentication. Well prepared speech, definitely worth watching.

Tom Stuart (@rentalcustard) gave a somewhat vague talk about simplistic, 3-legged construction of lisps interesting features of Smalltalk, Lisp and Bash, that surprisingly make them impractical. As to why that is – he left us with an open question. Or I might have gotten it completely wrong. Last but not least we had some fun with blinking arduinos and drons flying over the stage in the rhythm of psycho gangam style. All of that controlled by ruby programmed PS dancing mat with Julian Cheal jumping on it. Not everything worked as Julian had planned, but nevertheless the audience was in sheer joy for good couple of minutes. Perfect ending.

Dessert

Conference organisers have done a tremendous work making attendees (around 500?) feel taken care of. The auditorium was comfortable, lunch spots offered great Catalan food and evening parties were fun. Conference announcer Jeremy Walker also didn’t stay behind, even managing to do some live coding on stage.

My thanks go to all the heroes and Baruco team amongst them. Great job. See you next year!

Oh. And Barcelona, you too.

* Matz argumented that we should probably stop using MRI name for his first interpreter and switch to CRuby.

** Most Important Lessons

Feedback week at Lunar Logic!

Paweł Brodziński August 19, 2014

Our organizations would benefit if people shared more feedback with each other. In fact, I can hardly think of any company where sufficient peer to peer feedback isn’t an issue. On that account Lunar isn’t any different even though we treat transparency really seriously. We would appreciate more peer to peer feedback on a regular basis.

At the same time sharing feedback isn’t an easy task. Frequently it requires us to move out of our comfort zones. In the end, we just don’t do that.

Envelopes hanging on a blackboardOur solution for that was the feedback week. It is a very simple, safe to fail experiment. Anyone who wishes to participate tapes an envelope with their name on it to a wall in the common space and everyone else is invited to write down feedback for that person and put that in the same envelope.

To make it a really safe to fail experiment there are a couple additional constraints needed. In our case the whole thing was completely opt-in. If you didn’t require feedback no one forced you to participate. Also, only positive and supportive feedback was allowed.

That’s it. Told you. It’s simple.

Oh, and it lasted a little bit more than a week, that’s why we called it the feedback week.

Now, before I’d go further there was obviously a hypothesis behind the experiment. I assumed that if we provided an environment where we don’t force people to move out of their comfort zones they would share something new with others. At the same time as long as anyone put a single piece of paper in a single envelope I’d assume that it provided value in terms of generating more peer to peer feedback at Lunar.

One could question the rule about only positive feedback being allowed. One view on the usefulness of feedback is that it’s easier to build on ones own strengths than addressing weaknesses. From that perspective supportive feedback bears more value than critical information.

Personally, I don’t subscribe to that view. At least not as a general rule. I, for one, learn much more effectively from critical feedback. I do understand though that people have different learning patterns and for many supportive feedback is exactly what they need. For the rest, if not anything else, it feels good to hear good stuff about ourselves so there’s no potential downside really.

Another thing was that making the experiment safe was more important for me than maximizing feedback it produced. After all, we played with people’s behaviors and organizational culture. There’s no fallback strategy for such stuff.

Anyway, you want to like the results, right?

Almost everyone participated in the experiment. That validated the assumption that we strive to be getting more feedback. Then, talking about the volume of feedback, we could base it on the number of notes put inside the envelopes. There were a lot of them. It wasn’t just one here and there. People really did their homework and shared. A lot. (By the way, you guys are awesome! Thank you!)

Did it work?

Sure, it did. This isn’t the best part though. What I quickly realized was that many of us just used the feedback week as a catalyst to do the ultimate thing: go and share feedback face to face. One thing was that if someone wanted to go with something critical, well, the envelopes wouldn’t work because of the rules. Another thing was that some of us realized that they don’t need this artificial mechanism and they feel comfortable enough to share feedback the way it is supposed to be shared.

This was the real magic of the feedback week. It didn’t merely act as a one time event. It influenced our behaviors. And again, not only for a short while but in the long run. After all, once you learned that sharing feedback isn’t really as scary as you thought and people react to that really well, you will be much more likely do that again.

Oh, and by the way, many envelopes are still taped to the wall even though the feedback week is done. What’s more, since we are done with the experiment everyone sets their own rules, like “here goes everything that you can’t tell me face to face.”

Close-up of Dawid's envelope The best part about that tool is that it is applicable in pretty much any context. You can do that company-wide, but also in the context of a team or even individually. In fact, it would even work in a low trust environment. Of course results wouldn’t be nearly as good as in our case but you’d still get decent outcomes.

Even though I’ve pulled down my envelope I know there will be time when I’ll put it back for a while. I will be doing that till everyone feels comfortable sharing any crazy feedback they might have. It won’t happen overnight. In fact if it ever happens it would blow my head out.

Knapsack - faster feedback from CI nodes!

Artur Trzop August 5, 2014

knapsack logoA year ago we started working on a new project. We spend a few months adding new features and making old code better. The build was green and fast. The time flew by and our test suite grew – a natural thing, you would say. Our tests suite took about 12 minutes. When we were working intensively a few new commits in a repository causes long delay before we got feedback from our Continuous Integration server. That was annoying.

Balancing

An obvious thing happened: we decided to add extra CI nodes to split our tests across them. The simple solution for the split was assign an equal amount of test files per CI node. Some of test files, like unit tests, were super fast and others, like end to end tests, took much more. The simple split wasn’t smart. We ended up with three fast CI nodes and one very slow.

without_knapsack

It was sad seeing three CI nodes wasting their time.

Time is your friend

We tried a few solutions but calculating time was the best one. I started working on a gem called Knapsack. The name is based on the knapsack problem. :) This gem helps parallel specs across CI server nodes based on each spec file’s time execution. It generates a spec time execution report and uses it for future test runs.

Don’t waste your CI nodes’ time

Now with Knapsack our test suite is split across CI nodes in a more efficient way. Here is an example how time execution looks for each CI node with Knapsack.

with_knapsack

Get started with Knapsack

Add the gem to your Gemfile and run bundle command:

gem 'knapsack'

You need to bind the knapsack rspec adapter at the beginning of your spec_helper.rb:

require 'knapsack'
Knapsack::Adapters::RspecAdapter.bind

And the last thing, which is to edit Rakefile and add these lines:

require 'knapsack'
Knapsack.load_tasks

Generate time execution report for your spec files

After you add knapsack to your project you need to generate report with spec files’ time execution data. You should run the rspec command on one of your CI nodes.

$ KNAPSACK_GENERATE_REPORT=true bundle exec rspec spec

It will run all your specs and generate a file called knapsack_report.json. The contents of this file will be output at the end of the test suite. You then need to commit knapsack_report.json into your repository. Knapsack will use this file for better test balancing across your CI nodes.

This report should be updated only after you add a lot of new slow tests or you change existing ones which causes a big time execution difference between CI nodes. Either way, you will get time offset warning at the end of the rspec results which reminds you when it’s a good time to regenerate the knapsack report.

Using knapsack on your CI nodes

Run this command on your CI server where CI_NODE_TOTAL is the number of nodes, and CI_NODE_INDEX is how the CI server starts counting nodes (usually 0).

$ CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:rspec

The epic split: no problem

We are happier now because our CI feedback is much faster and we know that at any time we can add another CI node and have the epic spec split out of the box thanks to Knapsack.

There is always room for improvement

Do you want to help? There are a few things we can improve; like adding adapters other than RSpec or just improving the spec assignment algorithm. Feel free to fork Knapsack or just give us your feedback. Many thanks!

Oh, and one more thing, check the read me because Knapsack has even more features than described here.

a9n illustration I believe that many of you (ruby devs) have seen a file in one of your projects with with a lot of constants and full of if Rails.env.production?. It’s a nightmare. Even worse, it’s an example file that’s supposed to have a local gitignored copy. And because their content is usually not being verified, getting an uninitialized constant error with no clue of what happened and becoming annoyed is just a matter of time.

Another evil thing is the fact that all types of configuration items (access keys, mailer settings, etc.) are often split across various files and classes. Instead of having their dedicated place and being easily accessible, they mess up the code and their maintenance is a nightmare.

Unfortunately, Rails doesn’t offer any way to manage and verify custom configurations. Rails 4.1 introduces config/secrets.yml, which partially solves the problem, but it’s not enough to keep all the configuration maintainable.

I came up with my own solution called a9n (a numeronym for application) to keep my ruby and rails apps configuration easily maintainable, verifiable and clean.

Sources and gem

https://github.com/knapo/a9n

https://rubygems.org/gems/a9n

How it works?

a9n expects configuration.yml.example and/or configuration.yml file in the app’s config directory. You can have both – configuration.yml.example tracked by git and local configuration.yml ignored by git) – or just a single configuration.yml tracked by git.

If both files exist, content of configuration.yml is validated. It means that all the keys from the example file must exist in the local file, otherwise A9n::MissingConfigurationVariables is raised with information about missing keys.

All configuration keys are accessible by calling a method on an A9n. Let’s say you have:

defaults:
email_from: 'no-reply@knapo.net'
production:
app_host: 'knapo.net'
development:
app_host: 'localhost:3000'

So you can access the config by:

A9n.app_host # => `knapo.net` in production and `localhost:3000` in development
A9n.email_from # => `no-reply@knapo.net` in each environment

Custom and multiple configuration files

If you want to split a configuration, you can use multiple files. All files from config/a9n are loaded by default, but you may pass custom file paths as an argument to A9n.load, e.g. A9n.load(‘config/facebook.yml’, ‘config/mongoid.yml’). In such cases config items are accessible through the scope consistent with the file name. E.g. Having config/a9n/mandrill.yml:

defaults:
username: "knapo"
api_key: "1a2b3c4d"
production:
api_key: "5e6f7g8h"

You can access it by:

A9n.mandrill.username # => knapo A9n.mandrill.api_key # => 1a2b3c4d in production and 5e6f7g8h in other envs

Capistrano

If you are using use capistrano and you feel safe enough to keep all your instance (staging, production) configuration in the repository, you may find it useful to use capistrano extensions.

Add an instance configuration file e.g. configuration.yml.staging, configuration.yml.production (NOTE: file extension must be consistent with the capistrano stage) and add

require 'a9n/capistrano'

to your deploy.rb file. This way configuration.yml.[stage] overrides configuration.yml on each deploy. Otherwise you’d need to store configuration.yml in shared directory and link it in a regular way.

More details and the setup instructions are available on github.