Lift

The official Lift blog

Monday Jan 30 11am - 3pm Lift Office Hours with @dpp

Part of my ongoing commitment to Lift's growth and the success of Lift users and the Lift community, I will be doing "office hours" a couple of Mondays a month.

Office hours are an open invitation for anybody to drop by my office (541 8th Street in San Francisco) with Lift questions, suggestions, project demos or just to chat.

The first Lift Office Hours are from 11am PST to 3pm PST on Monday January 30th.

So, if you're in the Bay Area and want to chat, come on by.  There's plenty of coffee, tea, and other beverages.

Looking forward to meeting folks!

Thanks!

David

No, I don't owe you scala-tools.org

Apparently I'm a jerk for shutting down scala-tools.org.  Apparently, I'm an egomaniac for deciding not to sell the domain for "more than $0" even though nobody has made a legitimate offer for the domain. [Note: James Iry asked the question on Twitter.  It was a perfectly reasonable question that I answered as best I could in 140 characters. I answered him and there were subsequent posts from others that personally attacked me for not doing things the way they think I should.  Posts from others who attacked me for talking about using scala-tools.org to mourn the losses that I see in Scala-land.  This post is *NOT* aimed at James.  I like James.  I respect James.  James represents some of the very best of the Scala community and he was one of the folks who energized me about Scala and gave me hope that Scala could be a "local maximum of research and practical in computer langages."  I am deeply sorry that James read this post as something about him.]

I plan to transition the scala-tools.org accounts and such to another provider.  These plans will be rolled out on Tuesday.  There will be ongoing support for a Nexus/Maven repository for the Scala community.  The community will continue to have a place to share components.

What I do not have an interest in is selling or transferring the scala-tools.org domain.  There is not a way to set a reasonable price for the domain.  The domain has value because of the brand built over the last 3+ years of its existence.  How do you price that?  How do you price the value that the domain has brought to the Scala community as a whole?  Whatever I think that monetary value is, it's likely a few orders of magnitude more than others think the value is.  It's not even worth, in my opinion, trying to price it.

So, what value does scala-tools.org have to me if not a monetary value?  That's up to me.  Sorry, to say, but it's my domain.  It's something that I've paid for, worked on, recruited others to work on.  Yes, there's a conversation to be had with DavidB and Derek for their work over the years.  There's less of a conversation to be had with Josh because, while he helped out, he also dropped to ball on the Nexus skills transfer to Indrajit and Lukas which precipitated my decision to close scala-tools after Lukas expressed extreme frustration with being ignored.  I also owe some obligation to Indrajit and Lukas for stepping up when I put out the call for help a few months ago.

To the rest of the community, I owe a reasonable transition to a new hosting solution and that reasonable transition will happen.

But for those of you who have some notion that my contribution to the community over the years creates an obligation to continue to contribute, to keep giving, to keep doing unpaid, community service for you, you're not living in reality.  The fact that I have given my time and my effort and my cheer leading and my coding and my writing and my servers and time I could be spending with my family (like the first day of my kids' first spring break that I missed because I had to ward off a DoS attack against scala-tools.org and keep the system running) and all of that stuff does not mean I am forever obligated to keep giving my time, my money, my efforts, and the other stuff that I've given in the past.

My life goal is to leave things better than I found them.  I feel a fierce obligation to that who have relied on me, and thus when I make transitions, I try to make them gracefully so that there's plenty of time for others to make those transitions.

When I made the decision in late May to do my next startup, http://visi.pro, in a language other than Scala and on a platform other than the JVM, I made sure to have a graceful transition of leadership in the Lift community and I continue to support Lift and the Lift community because I owe an obligation to those who adopted Lift.  

When it became clear that Josh and Derek were not going be able to continue to support scala-tools.org, I put out a call for more volunteers.  Given that I was phasing out of the Scala community, I could have just shut scala-tools.org down then.  But I asked for help and worked on a transition.

But now, now when it's clear that it's time to shut scala-tools.org down, there's plenty of "you should do this," "you should do that."  Well guys, where were you when I put out the call for help a few months ago?  What has changed that would make you capable of making scala-tools.org work now?

More importantly, what gives you the right to insult me personally for making a choice about something I own and something I contributed to mightily?  If you wonder why I'm sad about the state of the Scala community, just read my blog, watch my Scala Lift Off London keynote.

My way of working through my sadness about opportunities lost, my way of mourning these losses will be expressed on scala-tools.org.  I will use something I had a mighty hand in building to express sadness about what could have been.

Will this cause some inconvenience?  Maybe.  Will this raise awareness about the issues holding Scala adoption back in a way that I've been unable to do in other ways?  Maybe.  Is this my choice?  Yes.  Do I owe you more than a smooth transition to another domain and another provider?  No.  Do I owe you scala-tools.org?  No.

Scala-tools.org winding down

Scala-tools.org has been running for more than 3 years, providing Maven repository hosting to the Scala community.

Scala-tools.org was initially hosted on a machine that I owned and paid for and was co-administered by me and David Bernard.  In May, 2009, we transitioned the hardware to something more robust as well as having Derek Chen-Becker and Josh Sureth take over the administration tasks.  I still own the machine and pay for the hosting and bandwidth as well as organizing the administrators.

In the 2nd half of 2011, Josh and Derek got busy with family and new jobs and a whole bunch of other stuff.  Łukasz Kuczera and Indrajit Raychaudhuri volunteered to help out and were hoping for a transition from Josh and Derek.  The transition did not go smoothly.

While it's probably possible to dig through the configurations and such that Derek and Josh put together to figure out how to keep Scala-tools.org running, it's not in my heart to do it or to ask that others do lots of work on scala-tools.org.

The Scala community should be serviced by the company that is making money off the efforts of the Scala community, Typesafe, rather than a bunch of people contributing thousands of dollars a year of hardware and bandwidth and time to do something that could be done by the commercial Scala entity.

I owe a debt of gratitude to David B, Derek, Josh, Łukasz, and Indrajit for their efforts on behalf of the community.

In terms of scala-tools.org, the server will continue to run for a while.  If Nexus or Jenkins needs a restart, I'll do that sort of thing.  We will not be accepting any new hosting requests.  We will not provide service to folks if configurations get messed up or if they need changes to permissions, etc.  I expect that the server will continue to host artifacts until mid-June of 2012 (I'm going on vacation in mid-June and will have spotty IP connectivity, so I can't be around to fix things.)  This should provide a reasonable period for projects to find new hosting.

Thanks.

David

Announcing Lift 2.4 Final

The Lift team proudly announces the availability of the final release of Lift version 2.4.

Lift is a powerful, secure and most matured web framework available today. There are Seven Things that distinguish Lift from other web frameworks.

Lift applications are:

  • Secure – Lift apps are resistant to common vulnerabilities including many of the OWASP Top 10
  • Developer centric – Lift apps are fast to build, concise and easy to maintain
  • Scalable – Lift apps are high performance and scale in the real world to handle insane traffic levels
  • Interactive like a desktop app – Lift's Comet and Ajax support are super-easy and very secure

Read an overview of how Lift achieves these important goals.

Lift open source software licensed under an Apache 2.0 license.

Lift 2.4 includes the following new features and enhancements:

  • Lots of enhancements to JSON support
  • Record improvments
  • Squeryl/Record support for Crudify
  • Singificant enhancement to MongoDB support (including better support for reference records and binary fields)
  • Support for BsonDSL (BSON types to JsonDSL)
  • Significant enhancements to Mailer functionality
  • Significant enhancements to the CSS Selector transformers
  • Enhancements to Snippet resolution like sub-packages based resolution, Loc-based snippet resolution of Screen and Wizard, etc.
  • Significant improvement to REST support including stateless Async/Continuations on Jetty 7, Jetty 8 and Tomcat/Glassfish
  • Ability to get html5 compliant templates using data-lift attribute
  • Numerous localization modules
A complete list of changes are available for all the preceding milestones here:

Please join the Lift Community and enjoy building awesome apps with Lift.


Thank you, enjoy Lift!
- The Lift Team

Scala's version fragility make the Enterprise argument near impossible

I have been working with Scala for more than five years.  In those five years, I've seen Scala evolve and seen the ecosystem and community evolve.

An attribute of Scala is that the Scala compiler generates fragile byte-code.  This means that all the code in an executable (JAR or WAR) must be compiled with the same library and compiler versions.  If you're using Scala 2.9.1 in your project, you must compile against a version of Lift that's compiled against Scala 2.9.1 and all the Lift dependencies must also be compiled against that version of Scala.  This is a reason that Lift has precious few Scala library dependencies.  It's also a reason that Lift is sprawling... there are a lot of modules in Lift that need to be cross-compiled and rather than having an external ecosystem of modules, we need to make them part of Lift to make sure they are all compiled against all the versions of Scala that Lift supports.

The by-product of this issue is that it's nearly impossible to test Lift or a Lift-based app against a pre-release of Scala.  This is one of the many reasons that there's almost always a "critical bug fix" release of Scala a few weeks after any major release.  The first chance that the broader community gets to use the ecosystem of Scala libraries and frameworks is after a release and after the cascading compilation/test/release 

A year+ back, during the 2.8 development cycle, some of us got together on the "Fresh Scala" project.  Basically, the idea was to have continuous integration of the major libraries in Scala-land and weekly milestones so that the community and the ecosystem could give better feedback on new versions of Scala.  Due to a lack of volunteer time, we never got the CI system built and here we are a year and a half later with the same Scala version fragility issues, but one thing has changed.

Scala is no longer exclusively an academic and community effort.  Scala the language and some associated libraries are now part of an enterprise-targeted suite from TypeSafe.  TypeSafe is a huge (20+ heads) venture funded company that is supposed to be bringing Scala to the enterprise.  But Scala's version fragility creates two huge costs for complex enterprise-type systems:
  1. All dependencies must be compiled against the exact same version of Scala and every referenced Scala library which makes in-house builds when there are multiple teams amazingly complex or time-consuming
  2. External libraries (like Lift) cannot have multiple layers of dependencies because you need to match the exact version of every library and Scala version all the way down
So, if you're in an organization with more than 2 or 3 development teams that are all generating internal Scala code, you can generally agree on the version of Scala and the various libraries that you're going to use.  When a new version of Scala or the libraries are released, you can get a couple of people in a room and choose to upgrade (or not.)  But once you get get past 3 teams, the coordination efforts become significant.  Trying to get 10 different teams to coordinate versions of Scala and Lift and ScalaZ and Rogue and Dispatch, etc. is going to be a significant cost.

As a library builder, the version fragility issue means that our library is not nearly as good as it could be and we can't service our community the way we want.

In terms of servicing the community, we cannot have predictable releases of Lift because we try to time our releases such that Lift will be released soon after a new version of Scala is released.  Given that Scala release schedules are not predictable and given that we need to wait for the dependencies (e.g., ScalaCheck and Specs) to be upgraded, we can't answer simple questions like "when will Lift support Scala 2.8.2?"

More importantly, there are few external Lift modules and they are generally not available except for specific versions of Scala and Lift and there are always issues when there's a version mismatch.  Modules like Rogue and Reactive Web are available for a small subset of Lift/Scala version combinations.  There are dozens of folks who have smaller modules for Lift, but none of them are cross compiled for the last year's worth of Scala and Lift releases.  This basically means you can use Rogue (something I'd suggest) or you can use Reactive Web (something else I'd suggest), but you can't use them in the same project because there's no guarantee that they are compiled against the same versions of Lift and Scala, nor is there any guarantee that they will be compiled against the versions of Scala and Lift that you want to use.

So, the Scala library and framework ecosystem is about as big as it can get given the Scala version fragility issue.  Because its not possible to have multiple layers of library dependencies in Scala, there will either be monolithic projects like Lift or islands of projects like ScalaZ.  And at this point, Scala does not have enough native libraries and frameworks to make it enterprise-worthy.  Yes, you can use Java libraries, but as others have found, the cost of wrapping Java libraries in Scala is often not worth the cost/effort.

What have I done about the issue?  Over the years, I've raised the issue privately and it has not been addressed.  I have tried to organize a project to address part of the issue, but haven't had the volunteer uptake for the project.  I have had discussions with Paul Phillips about what could be done and Paul, Jevgeni and I even worked out a system that could be integrated into the Scala compiler to address the issue in 2010 when we were at the JVM Language Summit.  The best answer so far is the Migration Manager, a closed source tool that may address some of the issues (I haven't tested it because I'm not going to use any non-OSS software for compiling or testing Lift to avoid any legal issues.)

What we need:
  1. We need something like Scala Fresh so that there's continuous integration across the major (and not-so-major) libraries and frameworks in the Scala ecosystem
  2. Something baked into the Scala compiler that makes the version fragility issue go away
What you can do:
  1. Blog about the issue and raise awareness about the version fragility issue and how it has impacted you
  2. Next time you're on a TypeSafe "what would you pay us money for?" call, please tell them that a vibrant Scala ecosystem is critical to wider adoption and the version fragility issue is significantly impacting both internal development efforts and ecosystem growth.  In order for your company to consider buying anything commercial from TypeSafe, you need broader internal adoption of Scala and that comes from reducing the transaction costs of using Scala including eliminating the version fragility issue in the open source parts of Scala.
Why am I writing this post today?  The version fragility issue has come up a bunch of times on the Lift list and the Lift committers list in the last week.  It's a non-trivial piece of pain and rather than being "go with the flow" about it, I figured I'd write about it and let folks know that it's pain and it's pain that could go away.

A Huge Thanks to Seaside

It was a little more than 5 years ago when I was searching around for a post-Rails web framework that I spent some quality time with Seaside.

Seaside is a Smalltalk-based web framework that provides power and simplicity by maintaining state on the server.  Web apps in Seaside are a continuation of a calculation.  Answers provided by submissions of forms allow the computation to continue.

I did a few months of work in Seaside and was working on adding Ajax to Seaside.  However, deploying a Squeak virtual machine and living my life in a Smalltalk VM were sufficient barriers that I went looking for something "more normal" for me.

However, the few months I spent with Seaside was tremendously influential in the way I approached web frameworks.  You can see echoes of Seaside in Lift's CometActor's ask/answer paradigm (the ask/answer thing even made it to Goat Rodeo.)

Seaside uses GUIDs on the client to mark state on the server.  This simple concept is bedrock on Lift and has served Lift very well in terms of allowing developers to express complex, interactive apps very simply.  The bedrock of GUIDs also leads to much of Lift's security.

So, this Thanksgiving, I'd like to give thanks to Seaside and Avi Bryant for inspiration and a template of excellence.

It was 20 years ago today...

It was 20 years ago today that I embarked on an epic computer journey that defined my life as a coder and a business-person.  On November 17th, 1991, I laid down the first lines of code for the Mesa spreadsheet.

 

Mesa was a spreadsheet for NextStep and continues (without my help) to be a spreadsheet for OS X (which is the updated version of NextStep.) At the time I started coding Mesa, the spreadsheet category on NextStep was surprisingly crowded with an entry from Lotus, the amazing, but non-traditional Improv, and WingZ from Informix.  Additionally, a venture funded company was working on traditional, native NextStep spreadsheet called PowerStep.

 

I built a company, Athena Design, while building Mesa.  We launched Mesa in August 1992 and within 6 months, Mesa was the dominant spreadsheet in the NextStep market.  While some called me a Lone Wolf, in fact Athena and Mesa were an amazing collaboration by a stellar team.  Put another way, I built Mesa and Athena with a little help from my friends (at this point, please start listening to the Beatles Sgt. Pepper’s)

 

What was amazing about Mesa was we built it based on talking to spreadsheet users in the NextStep community.  NextStep sold heavily into financial services and those folks wanted to be able to tick real time data from Reuters and other data feed services, perform calculations, and trigger external events.  Mesa did this.  In fact, working with IBM and Reuters in the late 1990s, I discovered that Mesa was the original art for the “Real Time Spreadsheet.”  Additionally, Mesa was an early example of a spreadsheet component that could be embedded in other applications (maybe it was the first, although my friend Debby Meredith apparently wrote a Smalltalk spreadsheet that could be embedded as well.)

 

While the Next, and subsequent OS/2, OpenDoc, Taligent, and Be markets were not large enough to support a growing company, what Mesa represented was that a small team of amazing people who listened to users and mixed in some of their own vision could fundamentally change the landscape.

 

Five years ago, when I discovered Scala and started coding Lift, I made the decision, once again to eschew the status quo of MVC and, instead, learn from what users and developers need.  For the last 4 years, Lift has been the best web platform for building Comet, Ajax, and Secure web apps.  Lift has grown and evolved, yet the core decisions about how Lift worked have remained correct and solid, and users like Foursquare and the Guardian agree.

 

Modeling Lift and Mesa on user-needs rather than machine needs is a different approach.  It’s an approach that centers the product on the user rather than the machine.  Sometimes the difference is subtle and sometimes it’s not, but in the end, and  as Kernighan and Ritchie taught us, people time will always get more valuable and computing cycles will always get cheaper.

 

Now, today, 20 years to the day after I started my epic journey, I am launching into a new journey: Visi.Pro.

 

Visi.Pro brings Cloud Computing to the rest of us.  Visi.Pro is HyperCard for the iPad.  Visi.Pro lets users build iPad/iPhone apps on the iPad (or Mac) and deploy them seamlessly across the Cloud including using Cloud resources such as persistence, data sources and data sinks.

 

Visi.Pro provides a complete environment from language to IDE to run-time to data/app/component emporium.  Visi.Pro provides a complete environment for the rest of us to create interactive, powerful apps.  A video of the vision, a vision statement, and more can be found at http://visi.pro.

 

Building Lift and the Lift community, I have realized that standing on the shoulders of giants is the best way to build something world-changing.  The best giants around as the folks who make up the open source world and a substantial portion of Visi.Pro including the language and much of the iPad code will be open source under an MPL license.  This means that if others want to build stuff off the Visi language, a language so simple and powerful that any Excel or PHP developer will be at home with it, they are welcome and encouraged to do so.  You can check out more about the language and such at http://visi.io Please check out the code and the community... please contribute back to the conversation and the code.

 

Building Visi.Pro and Visi.io will be a long, tough learning process.  It’s going to be interesting to learn from the PHP and Excel users in the world.  It’s going to be great understanding the kinds of things business people and moms and teachers want to build on the iPad.  It’s going to be like skiing double-diamond black slopes to build an invisible type system into Visi so that the users are always guided to do the right thing.  It’s going to be spectacular to get the Program Language Theorists to work through a type algebra (yes, Visi will allow an algebra in the type system) along side a teacher describing a grading app.  But what will really generate a tsunami of joy is when people are able to build amazing, interactive, real-time, cloud-backed apps on their iPad.  I can’t wait for that day and I know it’s going to be a long, tough, and spectacularly rewarding journey for all involved!

 

Before I ask you to join me on this journey... before I ask you to join the Visi conversation, I want to talk about some changes in me and how this will impact the Lift community.

 

For most of my career, I have aligned myself with the dark horses and the lost kittens.  I bet on an ousted Steve Jobs and his Next platform.  I bet on OS/2 and OpenDoc when Windows was a better choice.  I bet on Jean-Louis Gassée and BeOS because he told great stories about RepoDepo.  I bet on Java and WebLogic in 1996.  I bet on Scala.  Some of those amazingly risky bets worked out and others, well, didn’t.  This time, I’m betting on the clear winner.  It’s clear that iOS will dominate the OS/computing landscape for the next generation.  It’s clear that the cloud and interconnected and partially connected computing will be the way it’s done for the rest of my professional life.

 

So, rather than choosing the dark horse, I’m going to bet on the winner.  And rather than choosing an existing category and simply building the best technology for that category, I’m going to create a new category (or more properly revive a lost category... the category of “everyone can program this machine and make it beautiful” category.)  I’m going to fundamentally change the way people program for the iPad and the Cloud. I’m going to kick a dent in the universe.

 

My commitment to Lift remains.  I love Lift.  I love the Lift community.  I remain in awe of the amazing committers.  I am stunned and amazed by the folks who have committed to using Lift and I remain committed to Lift and the folks who have adopted Lift.  For most of 2011, I have spent about 12 hours a week in the Lift community... working on the mailing list and closing tickets.  Most of this year, I’ve been doing Lift-related consulting and much of my contributions to the Lift code-base has been for my clients.  I will continue to spend 12 hours average a week in the Lift community.  What is going to change is that the tickets I close, the code I write for Lift, is going to be entirely community-focused.  It’s going to be based on what the community needs rather than what my clients need.  Longer term, I will find a way to make Visi and Lift play together.  They are both my children and I love them... although sometimes newborns need a little more attention than 5 year-olds.  The key change is that I will not be available to consult on Lift-related projects in the future.

 

So... what do I want from you?  

 

I want your brains.  I want your input.  I want your language ideas.  I want your app ideas.

 

Are you involved with Rust, Roy, Haskell or other amazing languages?  Have you built stellar iPad apps like Codea and Blueprint?  Please join the conversation.  We can share ideas and together be more than the sum of our individual parts.

 

Are you someone who wants to build something on your iPad?  Something beautiful and amazing?  Something that you can share via the cloud with others in real-time?  Something that sucks data from SalesForce, Twitter or a million other data sources or sinks on the Internet?  Please join the conversation.

 

Visi.Pro is going to be a totally epic journey and I look forward to you joining the team and making Visi.Pro life-changing for millions of iPad users.

 

Thanks!

 

PS -- Visi.Pro and Visi will always be defined, developed and built with strict adherence to Apple’s policies about what it allows on the App Store and on the iOS.  Visi.Pro, like the amazing Codea or Numbers, is a development environment and interpreted runtime.  I welcome any public or private (at Apple’s option) conversation with Apple regarding Visi.Pro to clarify the iOS and App Store rules and to insure that what we do with Visi.Pro is done clearly on the correct side of the line and with Apple’s happy blessing because, at the end of the day, our goal is to empower iPad and iOS users to rock their respective worlds!

Announcing Lift 2.4-M5

The Lift team is excited to announce the availability of Lift 2.4-M5 (Milestone 5).

Quite a few enhancements and bug fixes have been added since Lift 2.4-M4 for this release.

Lift 2.4-M5 would be the last milestone before 2.4 final release. If your project depends on 2.4, now is the time to test things even more rigorously.

And, as usual, join the Lift community and enjoy coding with Lift.

Thank you, have fun!
- The Lift Team

Grip goes live with Lift

Grip is live with their systems built on the Lift web framework.  Grip is a financial decision support tool that enables financial institutions of all sizes to brand and offer an app that provides the most sought after mobile services. It aggregates consumers' balance and transaction data from unlimited number of domestic financial institutions into a single, up-to-the-minute collective view, empowering them to understand their money flow simply and without clutter. With no core integration, no set-up fee, no annual fee and no minimum user agreement, Grip simply cost the financial institution .99 cents per active user, per month.

It's very exciting to see more companies basing their business on Lift... especially companies from Iowa.