Some notes on the future of Haskell and FP

Don Stewart, Duncan Coutts and Simon Marlow ended the Haskell Symposium yesterday with a wonderful session about the future of Haskell.

Don and Duncan began with an acknowledgement of the importance of libraries to the health and future of the Haskell language community. A long-standing need for a central source of good libraries was met about 18 months ago with the development of the Hackage package database. Since its introduction, we have already seen roughly 750 open source Haskell packages published.

As an indication of how effective Hackage is at making leading-edge research available quickly, Don cited Jean-Christophe Filliâtre’s functional implementation of the Garsia-Wachs algorithm (paper, slides). He presented this work at the ML Workshop on Sunday. Nicolas Pouillard ported the ML code to Haskell a day later, and published it on Hackage immediately.

The next piece of the library puzzle has two motivations:

• to get the developers of GHC out of the business of library maintenance, so they can focus on compiler hacking; and
• to broaden the set of libraries that Haskell hackers can depend on their OSes and distros to provide.

Don, Duncan and Isaac Potoczny-Jones are working on a library platform for Haskell. This will begin life based on the current extralibs bundle, currently a grab-bag of somewhat random libraries. As the platform evolves, it will undergo milestone releases every six months, and the set of libraries that it includes will broaden over time. Libraries will be added based on their popularity and quality.

The idea for the platform is that as a Haskell developer, you’ll be able to download a Haskell platform release on a specific date and get a family of stable, useful, liberally licensed, mutually compatible, and portable libraries that you can immediately use to develop applications in.

Downloading and installing the Platform, and other libraries, is now a simple task, based on Duncan’s hard work on the cabal-install command. It downloads, builds and installs both the library or app and all of its dependencies given a single command, “cabal install myPackageName”.

Developers can voluntarily report build results back to Hackage, which subsequently publishes them. This lets you glance at a package’s build reports before downloading it, to see whether anyone has had success with it on your platform before.

Uploading a new or updated library to Hackage is almost as simple: build a tarball with “Setup sdist”, then upload it with “cabal upload dist/myPkg-1.2.tar.gz”.

Over the coming year, the Hackage developers will be extending the features of the central package database, to include voting and feedback features. This will help users of packages to determine in advance whether they’re getting solid code, and to share any good or bad experiences they’ve had.

Simon Marlow gave a very encouraging talk about the future of the Haskell language standardisation process, Haskell Prime. In spite of sterling work on his part and a substantial amount of work from others, he estimates that after three years, the process is perhaps 10% complete. Rather than get mired in gloom, the committee met a few evenings ago and came up with an outline for a change to the process.

The fundamental problem with the old process was its goal: to produce a complete revision of the language definition. Given the large number of activities that each committee member was participating in, it was simply not going to be possible to get them all together for a long enough period to settle all of the proposals.

Instead, the intention is now to move to a more modern style of development: to issue annual addenda to the language standard, each containing incremental changes to the current language definition. The details have not yet been decided upon, because only about half of the committee members are at ICFP, but Simon and his colleagues will be working to refine the proposal over the coming months.

Simon then changed tack, and discussed some of the changes that we can expect to see in GHC over the coming 18 months. Much of this work will focus on making multicore performance better and more consistent.

John Dias (a student of Norman Ramsey) is leading an effort to replace GHC’s back end and code generator. This is particularly exciting, as the new back end will be based on top-notch dataflow optimization research, and there’s clearly plenty of scope to improve the quality of both scalar and numeric code.

GHC 6.10.1 (now in public beta) introduces two promising changes for multicore programs:

• Simon’s new parallel garbage collector. This is still a stop-the-world collector, but it will collect the heap using multiple threads.
• Manuel and his colleagues have a technology preview of Data Parallel Haskell. If you program to the library APIs, it’s possible to get good parallel speedups now, but the vectoriser is a bit broken, so parallel array comprehensions don’t work so well.
• Some performance limits on GHC’s threading support have been eliminated: it can now run a million or more threads concurrently, without breaking a sweat.

What further changes can we expect to see?

• Simon intends to work on concurrent garbage collection, using per-CPU nurseries. This will move us away from the current stop-the-world model, and should make an enormous difference to performance and scalability on multicore systems.
• The par combinator needs an overhaul: its activation cost is currently too high to let us sprinkle it about liberally.
• Parallel profiling will help to characterise the performance and scaling properties of parallel programs, and to identify the sources of space leaks that can sometimes be introduced with par.

It’s hard to overstate how exciting all of these changes are.

• Libraries breathe life into a language. Between the Platform initiative, the continued growth of the library community, and the refinement of Hackage, Haskell is quickly acquiring a stable, high quality family of libraries that developers can depend upon.
• The language standardisation process is about to become sustainable, and will no longer be logjammed. This means that popular language extensions will start to become available in other compilers, which should foster an increase of friendly competition between compiler development teams. Facilitating competition is both exciting and important: huge strides were made in performance during the early 1990s when there was a three-way race between GHC, HBC, and the Yale Haskell compiler.
• With its foci on data parallelism, improved code generation, better profiling, and multicore performance and scaling, GHC is the most exciting vehicle for innovation in high-performance parallel and concurrent code.

As a Parthian shot, I have been astounded at the number of people at this year’s ICFP who come from outside academia. Many of these people aren’t even using functional programming at work: they came out of curiosity; and because they wanted to get involved in the community. They are all tremendously excited about the energy and intellectual ferment involved, and at the welcome they’ve received from the academics. And talking to the academics, they’re thrilled to now be the focus of so much interest.

There has been a marked change in the tone of ICFP recently, and it is in my estimation by far the most exciting annual conference in technical computing. The next five years are going to be absolutely tremendous for functional programming.