Saturday, November 3, 2007

RubyConf 2007: Keynote by Matz

Part I Language Matters or Not

Turing Theory => Language does not matter, at least in theory. Twitter On Rails. Trouble illustrated by a lolcats photo. There has been a 1000% performance boost since January 2007. What made the difference? MySQL, Solaris and dtrace.

What makes the difference? Not the language itself. It's the attitude of the users. Python users believe in the one true way. Ruby users like diversity. I said that I liked all programming languages, except for a few. As proof, Matz is wearing a python shirt.

People matters. Motivation Matters. Enthusiasm Matters.

Martin Fowler said, "Ruby people are nice." There is a Love/Hate Ratio graph. Ruby, Lisp and Python are the top three. We are using the most beloved language in the world.

Thank you!

Part II What's in 1.9, or What's not in 1.9

We will be releasing 1.9 this Christmas. Unfortunately it is not as stable as we would like. The Grand picture is that 1.8 is the long term stable release, 1.9 introduces major incompatibility and will be the bleeding edge for a time, and 2.0 is the driving force.

1.9 Incompatibility

Some will be severe, and some will be trivial. Severe #1: block arguments are changed. They are local variables only. Blocks are scoped more strictly. Severe #2: string is no longer enumerable. What will be iterated over needs to be specified explicitly (lines, chars, bytes). Severe #3: M17N Unicode is introduced. Character-based indexing ("ABCaio"[0] returns "A" and [3, 1] returns "a"; the preceding "aio" should be read as Japanese Hiragana, because I don't have Japanese support on this computer). Encoding aware comments. You can specify UTF-8 in a special comment at the beginning of your script. You can open files of other encoding formats.

Trivial Incompatibilities: lots of changes, some deprecation, and even more enhancements. Some new features:
  • "->" lambdas ("->" is a mutation of the lambda character)
  • ".()" to call
  • multiple splats in an argument list
  • mandatory arguments after optional arguments
  • built-in enumerator
  • external iterator (e1=[1,2,3,4].each e2=[10,11,4].each loop{p e1.next+e2.next} #prints 11, 13 and 7; loop will catch the end of iterator silently; inherited from the language named Satan?)

Part III Implementations

Realistic alternatives: JRuby, Rubinius, and IronRuby. All are faster than MRI. Finally, we succeeded in getting rid of Matz (It's ok; I'm a designed of languages). Also, we have YARV, the fastest Ruby VM ever. Praise should go to Koichi. In 1.9, Ruby goes on a major memory diet.

Part IV Where are we from? Who are we? Where are going?

Where are we from? Many languages.
Who are we? Rubyists.
Where are going? World domination.

Beware of a bubble economy and the cycle of tragedy (humble start, success, pride, conflict/war, drop to nothing, repeat). We are at success.

Where will we go? Enterprisey. I don't want to, but I can't help it. The suits are surrounding us.
Where will we go? Scalability. It is not a language issue. It is a system issue (data size, access frequency, program size, and number of CPUs).

Q & A

1.8 has green threads. Will 1.9 have native threads, due to the move towards multi-core CPUs? I envision multi-core CPUs will use multiple processes.

No one is working on a shim library for 1.8->1.9 as there was in 1.6->1.8, because of the massive differences.

What will required arguments after optional arguments look like?
def foo(a, b=24, c)
...
end
foo(1,2,3) # a =1, b=2, c=3
foo(1,2) # a=1, b=24, c=2

def bar(a, b=1, c=2, d)
...
end
bar(1,2,3) # a=1, b=2, c=2, d=3


You talked about named arguments at another conference. What is the status on that? We didn't have enough time to complete that for Christmas. You will have to wait until 2.0 for that.

Why would you want the ability to have required arguments after optional arguments? It is useful with named arguments. Fill the required arguments at the front of the list, then at the end of the list, and finally fill in the optional arguments.

Are you worried about the niceness scaling as the community grows exponentially? I was worried when the conference went from 60 people to 500 people. I think it has been a success so far. It is kind of like a miracle.

With the external iterator throwing an exception at the end of the loop, is there a stack trace associated with it and is it a performance consideration? Someone from twitter answered that the bug they found had an exception thrown with a 400 level stack trace. If you aren't doing anything crazy with the iterator, you should be fine.

Will the 1.9 byte code be frozen anytime soon? Deferred to Koichi. No.

Will there be an AOT compiler in 1.9? Deferred to Koichi. Not in the Christmas release. Sometime in the future.

What would you like to see in Ruby 3.0? 3.0? Class box. We should be able to isolate our changes to a class in a name space. Looking to do that in 2.0.

Why does method wrapping (aspects) keep getting punted? I feel that the features in Aspect/J is application-level or maybe framework-level. The language should provide primitive features.

How will future pressures of forking be dealt with? Fortunately or unfortunately, Ruby is difficult to implement. People implement lisp once or twice. I implemented Ruby once; never again. Ryan Davis shouts, "How many people have implemented lisp?" 10% of the room raises their hands.

Have you given any thought to passing multiple blocks to a method? I tried several times, but was never successful. It is important to keep the simple syntax that the Ruby block has.

Speaking of crazy syntax, do you like list comprehensions in Haskel? No.

That's all folks.

Disclaimer: I might have misheard any portion of this.

RubyConf 2007: Conversations vs. Laws: What do we mean when we say Ruby is dynamic?

Francis Hwang is a member of NYC.rb

This talk is meant to be provocative. He gives a forewarning, "I am not comparing Java programmers to slave holders."

For the sake of discussion, types are abstract, and classes are concrete.

Platonic idealism. There are the things you see in this world, and there is something abstract behind that. A specific palm tree or evergreen tree vs. the idea of a tree.

Linnaean taxonomy. The different categories are up to debate, except for species. That category is testable: two things are in the same species if they can interbreed. Exception to the rule: ring species (laurus gulls).

He goes into race and taxonomy as a comparison of type and class.

Does anything that matters have a fixed definition? Are types real? No.

Now to the code ...

array.rassoc(key) "Is this a list? Is it a stack? Is it a map?" (Elliotte Rusty Harold) Type doesn't matter to dynamic language programmers.

Ruby is a low social contract language (no spec).

...

He brings up quotes of dysfunctional companies that have broken feedback loops.

Toyota Kaizen Approach.

Programming team ability versus results. Businesses who talk about core competency look at only a small slice of this curve. Is it possible to hire a crap Ruby team that will ruin your company? Absolutely.

Nothing is permanent.

RubyConf 2007: Maximizing Productivity (or How to Hack Well)

Most of Eric Hodel's six years of Ruby experience is mostly in open source, covering about forty projects. He is a member of the Seattle Ruby Brigade.

Two key elements to what Eric does:
  • Have fun.
  • Cartman: Whateva! I do what I want!

Coding and Testing

Be selfish; fix your problems first. KISS (keep it simple stupid) & YAGNI (ya' ain't gonna need it). YAGNITAGAFI (... they ain't gonna ask for it). Test. Autotest. Keep everything in sight (code, test, autotest, and terminal all in one screen, with chat window in the gutter). Heckle. Automate everything you do (automatic setup, migrations, common tasks). Know your editor intimately. Pair Programming.

I'm seeing repetition from the Seattle Ruby Brigade member's presentations.

Releasing

Documentation (README and rdoc). Bug Tracker (no anonymous reporting because there is no possibility of feedback). Review (get constructive criticism from someone else). Pre-Hoe was not fun. Hoe (packaging, uploads, emails, and blogs are automated). Release early and often.

Relax

Balance out your life.

Maintenance

Bug Tracking. Gardening (iterative with simple goals and small changes,; keeping it well organized). Some people refuse to use a bug tacker. Grep through ruby-talk and sphincter? Search technorati. Be responsible (file good bug reports and be responsive to the developer). Versions (don't care, but be consistent). Know when to break up your code into seperate packages.

Inspiration

I write software for me. I don't owe you support (this second). Criticism (My software sucks. That's ok. People complain. That's ok. Maybe they'll be motivated to fix it.) Hacking Night is helpful. Pain-killers (automate something you do by hand).

RubyConf 2007: Tightening the Feedback Loop

Phil Hagelberg belongs to the Seattle Ruby Brigade.

Slides link

I left the other room because the topic was misleading and I wasn't really interested in what it turned out to be. The talk was originally called, "Writing Client and Desktop Applications in Ruby," and was changes to "Deployable Ruby Runtimes." The talk was about release a wholly standalone ruby application.

Testing
  • good - manual tests
  • better - test::unit / rspec
  • better still - autotest
  • a bit better
    • Flymake was released with the last emacs. It throws your buffer at gcc with warnings turned on. There is a port to ruby that throws your code at ruby -w.
    • test-unit.el gives you the some info from autotest, but provides better visualization
You want to tighten the feedback loop of breaking something and being notified of that breakage. You care about your code's accuracy. You use/write tools to help you measure your code's accuracy. You are notified when things go wrong. As you go down the list above, the delta between breaking something and being notified about it goes down.

What to measure?

Accuracy - syntax highlights, indentation, and test/specs. The first two happen automatically in the editor. Tests/specs should be shown there too.

Meta-accuracy - how accurate are your accuracy checks? Use rcov (caveat: don't think this means you are fine) and heckle.

Maintainability - Martin Fowler said, "Any fool can write code that a computer can understand. Good programmers write code that humans can understand." It can't be programmatically measured, but it is correlated to complexity, which is measurable. Use flog (caveat: the scores are relative). How to use it? Measure on a per-commit basis (no patch bombs).

Performance - black box response times (Julian? had a presentation at the last RailsConf that dealt with this. The code was about 50 lines. You should write your own.) and measure test suite times

This talk isn't giving me anything that I didn't hear in Ryan Davis' talk.

He kept bringing up how often do you want to be notified, in each of the categories.

Keep a history of your measurements, so that you can chart your personal growth.

The pragmatic programmer says to learn your editor. You should also learn what you can do for it.

There's a demo of Augment being used in emacs.

RubyConf 2007: Profiling and Tuning Ruby 1.8

Ed Borasky

Slides

The project name on RubyForge is Cougar.

Things you will not hear Ed say:
  • Premature optimization is the root of all evil
  • Dynamic languages are inherently slow
Is Ruby 1.8 slow?

Benchmark and collect the results. Settle on a common unit. Take the geometric mean (n-th root of (A1 * A2 * ... * An)) of the numbers. Alioth Shootout Results.

Hemibel Thinking

The art of only caring about ratios greater than a hemibel (half of a bel (magnitude of an order of 10) = sqrt(10) ~= 3). Modifying the Alioth Shootout Results to deal with hemibels, shows that in practical terms, perl, yarv, python and php are about the same speed. "Ruby is sort of slow."

But that's an oversimplified answer. Is there a "geometric standard deviation"? Yes, but box and whisker plots (boxplot) are more interesting. Yarv shows very little variation, followed by python and ruby. Although, yarv has some outliers.

Ok ... it's sort of slow ... now what?
  • Throw hardware at it?
  • Wait for 1.9?
  • JRuby?
  • IronRuby?
  • Rubinius?
  • Cardinal/Parrot?
  • Tune Ruby 1.8?
Let's tune it
  • Collect a benchmark suite
  • Profile the Ruby 1.8 interpreter on the benchmark suite
  • Identify the bottlenecks
  • Remove them
Ruby 1.9 has a benchmark suite included that will let you compare yarv with your installed ruby. There is also pet store from thoughtworks. The JRuby team is using it. gprof is one of the earliest profiling tools. The output of all the runs on RubyForge site: cougar/ProfilingAndTuningRuby/benchmarks_profiles. Surpisingly, most of the time executing Ruby is spent in the interpreter and not the actual work being done. Ed's added benchmark test of doing Matrix operations in rational arithmetic, rather than in floating point. The function that returns the remainder of bignum division was nowhere near the top of the list (sorted by relative time spent in function).

Using gcc/gcov, he does some lower level analysis on the function where most of the time is spent. There is a switch statement that has 86 branches. The top six or seven branches cover 77% of the time spent in this function. The interesting thing is that these are not the first six or seven branches in the switch statement (they are actually near the end).

Acovea is a genetic optimizer over GCC compiler flags. Although, it is unlikely to do better than "march=athlon64 -O3".

This is the second time I've been in an afternoon session where I've heard applause coming from the other room, making me wonder what I am missing. The other time was while I was in the Ropes talk.

He delves into some more advanced techniques.

This was an interesting talk from an academic perspective. This talk was more for the ruby core team.

Have you tried profiling it with valgrind and cachegrind? No. He mentions code analyst.

Pair Programming Question

I asked Ryan Davis, "How do you deal with scm check-ins when pair programming? Do you have n-choose-2 logins?"

His response was, "The person driving usually checks the code in. Sometimes we note who we were paired with, but we usually don't care."

I didn't ask him what his thoughts were about sharing one login among the group, but I don't think he would have approved.

RubyConf 2007: Rubinius 1.0

What is the end game for Rubinius, JRuby, IronRuby, or any other the other implementations?
Total
world
domination
For Ruby!

Rubinius effectively began at last year's RubyConf, in Denver, Colorado. Rubinius comes from SmallTalk, with Ruby syntax and behaviors added. Also, throw in some crazy cs papers to the mix.

CS Nerd Attack Mode! (blitznerdo) Zeropunctuation technical details.

Runtime and Kernel
  • MRI 1.8 - 80k lines of C and 0 lines of ruby
  • MRI 1.9 - 120k lines of C and ~50 lines of ruby (prelude.rb)
  • IronRuby 1.1aplha1 - 48k lines of C-Sharp and 0 lines of ruby
  • JRuby 1.0.2 - 114k lines of Ruby and ~1000 lines of ruby
  • Rubinius - 25k lines of C and 14k lines of Ruby
Rubinius is Ruby for Ruby programmers. Eating your dog food (using the tools you are developing) provides a feedback loop.

Who gets paid to write Ruby on a daily basis? About 90% of the room raises their hands. Someone form the audience asks, what about only those writing Ruby and not Rails? About 25% of the room raises their hands.

1.0 should be done about now. Why isn't it done? Worried about architecture and design decisions made now, that will affect the long term growth of the project. Mistakes were made.

His talk is focusing on the community, rather than on the technology. Rubinius has been able to garner weekend patchers and code-junkies alike. Free flowing commit bit (provide one patch and you get open commit rights) was borrowed from the perl community.

Engine Yard is funding Evan Phoenix.

Backtraces are cleaner than with MRI. Rubinius also gives you stacktraces for segfaults in C extensions.

Demo about MethodContext, Continuations and spaghetti stack. Do you plan on making the MethodContext serializable? Yes.