Links
Tags
apache
armenia
books
bsd
c
c++
chips
cinema
concurrency
cooking
database
dragonfly
erlang
filesystem
freebsd
fun
hardware
java
javascript
json
languages
linux
lyric
mac_osx
mail
math
misc
music
personal
poems
presentation
programming
python
references
ruby
rubyjs
scm
software
spiking_neural_net
study
sysadm
sysarch
technology
testing
travel
virtualization
web
wee
windows
Around 4 years have passed since the last release of Wee.
What is Wee?
Wee is a Seaside-inspired 1 web-framework for building highly-dynamic component-based web-applications.
What’s new?
- 100% Rack based
- Support for continuations
- JQuery AJAX support
- Ruby 1.9 ready
Distinctive Features
Wee is not just another web framework. It is completely different.
Continuations
You can use continuations to model the page flow of the your application, in the same way as you’d call a method in a GUI application. Continuations are optional; nothing in Wee depends on them. Example:
# you can write code like this:
if callcc YesNoMessageBox.new('Really?')
# do something
else
# something else
end
Without continuations one has to use (ugly) Continuation Passing Style (CPS):
call YesNoMessageBox.new('Really?') do |res|
if res
# do something
else
# something else
end
end
Backtracking
Backtracking means that the user can naturally use the browsers back button despite Wee’s statefulness. This is not taken for granted for stateful web frameworks. Behind the scenes, Wee keeps multiple states of the application around with only little help by the programmer.
Components
Contrary to the widely used model/view/controller (MVC) paradigm, Wee tighly couples the controller and the view within a component. Components itself are highly decoupled from the rest of the application and can be reused to construct more complex applications. The powerful programmatic HTML rendering approach further reduces wasting the programmers mind by avoiding switching files (controller/view) or languages (Ruby/HTML). Therefore in Wee everything is written in Ruby. The programmatic HTML renderer is not just a simple HTML builder, it provides very powerful methods for easily generating HTML constructs and registering callbacks. Example:
#
# Generating a <select> tag
#
# select an object from these items
items = [1, 2, 3, 4]
# the labels shown to the user
labels = items.map {|i| i.to_s}
# render it
r.select_list(items).labels(labels).callback {|choosen| p choosen}
# render a multi-select list, with objects 2 and 4 selected
r.select_list(items).multi.labels(labels).selected([2,4])
Call/Answer mechanism
From callback handlers you can call other components which in turn replace the current components view. The called component can later return (or answer) back to the calling componenent. Example:
class YesNoMessageBox < Wee::Component
def initialize(msg)
super()
@msg = msg
end
def render(r)
r.bold(@msg)
r.form do
r.submit_button.value('YES').callback { answer true }
r.space
r.submit_button.value('No').callback { answer false }
end
end
end
# Use call (or callcc) as in the Continuation section above
# to call a component:
call YesNoMessageBox.new('Really?')
The classical Hello World Example
require 'rubygems' if RUBY_VERSION < "1.9"
require 'wee'
class HelloWorld < Wee::Component
def initialize
super
add_decoration Wee::PageDecoration.new(title="Hello World")
end
def render(r)
r.h1 "Hello World from Wee!"
r.div.onclick_callback { p "clicked" }.with("click here")
end
end
Wee.run(HelloWorld) if __FILE__ == $0
# Now point your browser to http://localhost:2000/
Installation
gem install wee
More Resources
-
http://rubyforge.org/projects/wee
-
http://www.ntecs.de/projects/wee/doc/rdoc/
-
http://github.com/mneumann/wee
The last couple of days I spent refactoring my web application framework Wee, a Seaside-like framework for Ruby that I started back in 2004 with a lot of mental help from Avi Bryant (one of the main guys behind Seaside). It’s now approaching a 2.0 release. Wee is now fully Rack based (Rack is a commonly used Ruby Webserver Interface) and the code is in general even cleaner than it was before (huge parts were rewritten). Plus one cool feature: Continuations.
Continuations
Yesterday I finally thought it’s time to put back in continuations mainly because of some very interesting developments (i.e. patches) for Ruby 1.8.x, which seems to fix memory leaks that occured when using continuations.
Continuations in Wee were never used as extensively as they were used in Seaside. And it’s interesting to read that Seaside 2.8 reduced heavy usage of continuations – something that Wee did from the beginning :)
An example says more than thousands words, so here we go:
require 'wee'
class Page < Wee::Component
def initialize
add_decoration Wee::PageDecoration.new('Title')
super
end
def render(r)
r.anchor.callback {
if callcc YesNoMessageBox.new('Really delete?')
callcc InfoMessageBox.new('Deleted!')
else
callcc InfoMessageBox.new('Deleted action aborted')
end
}.with("delete?")
end
end
class InfoMessageBox < Wee::Component
def initialize(msg)
@msg = msg
super()
end
def render(r)
r.h1(@msg)
r.anchor.callback { answer }.with('OK')
end
end
class YesNoMessageBox < InfoMessageBox
def render(r)
r.h1(@msg)
r.anchor.callback { answer true }.with('YES')
r.space(1)
r.anchor.callback { answer false }.with('NO')
end
end
Wee.runcc(Page)
The interesting part of this example is the anchor tag callback handler of component Page. When clicked it will display a YES/NO message box which will return true if you clicked on YES, otherwise false. When it returns, execution resumes exactly at the point after the callcc call. In fact, callcc returns the return value of the called component. A called component returns by calling the answer method (it behaves mostly like the regular return statement). The concrete mechanism how this all works out is a bit more complicated as it involves installing and removing several decorations to delegate rendering and catch exceptions upon answer, but this is totally unrelated to continuations.
So how would it look without the use of continuations? Well, the callback handler would turn from:
if callcc YesNoMessageBox.new('Really delete?')
callcc InfoMessageBox.new('Deleted!')
else
callcc InfoMessageBox.new('Deleted action aborted')
end
into:
call YesNoMessageBox.new('Really delete?') do |res|
if res
call InfoMessageBox.new('Deleted!')
else
call InfoMessageBox.new('Deleted action aborted')
end
end
Actually in this concrete example it’s not hard to rewrite the code without using continuations. But for more complex examples it would get much worse. For example, using continations, a simple sequential flow like the following one:
callcc page1
callcc page2
callcc page3
would turn into the much less readable equivalent using CPS (“continuation passing style”):
call page1 do
call page2 do
call page3
end
end
Which one would you prefer? The good thing: In Wee you can use both, that’s why I have the two methods call and callcc, the latter not to mix up with Kernel.callcc.
Performance and Memory Usage
The reason why continuations were basically unsupported in Wee for a very long time was that they leaked memory. Since a few months there circulated patches on the ruby-core mailing list that seem to fix those leaks. Actually the reason seams to be that some parts of the stack are not overwritten when calling a function and as such old values keep referenced: a leak is born! I haven’t tried those patches, but I think they work. Instead I tried Ruby 1.9.1. And wow! It’s incredible! Memory usage stays constant at 12 MB, regardless of the number of requests. And performance scales nearly linearily as I increase the number of threads.
For 10000 requests (with one thread) Ruby 1.9.1 takes 16 seconds and requires 12 MB of memory. The same example with Ruby 1.8.7 grows to 329 MB of memory and takes 56 seconds. That’s an increase in performance of factor 3.5 and a 27-fold reduction of memory.
The example I’m using for the benchmark is the following 2-level nested callcc component call:
class Benchmark < Wee::Component
#
# calls Called2 then returns
#
class Called1 < Wee::Component
def render(r)
r.anchor.callback { callcc Called2.new; answer }.with('back')
end
end
class Called2 < Wee::Component
def render(r)
r.anchor.callback { answer }.with('back')
end
end
def render(r)
r.anchor.callback { callcc Called1.new }.with("click")
end
end
Conclusion
Continuations seem to be stable enough and not too expensive in terms of memory and performance in Ruby 1.9 so that there is nothing against using them (wisely) within Wee. This makes Wee the only web framework for Ruby to my knowledge that uses continuations. Coupled with other great features provided by Wee, for example reusable components, backtracking or the programmatic HTML generation, this undoubtly allows Wee to be called one of the most advanced web application frameworks for Ruby. Worth to note is that Wee does not focus on RESTful multi-million page-view serving, scaling applications. Instead Wee focuses on very complex applications, similar as found in traditional GUIs, and to get the job done quick and beautiful.
The last day I hacked on another piece of Erlang code. I gave it the name http_hub. It’s not that easy to explain what it actually does. Clients can connect to it, as it behaves for them like a HTTP server. The client’s request is forwarded to a backend server. But no answer is sent back to the client. Instead http_hub waits on another port (e.g. 14000) for another HTTP request from the backend server (usually a single worker node in a cluster will answer), which includes the response that should be sent back to the client. Once this arrives, the client is answered with the included response. If no request on port 14000 arrives within a specified time, an error response is sent back to the client.
The code is here.
During the last two hours I implemented a hyper-fast (4000 files with ~110 MB in total in around 4 seconds) link extractor. It reads file names of HTML files from stdout, extracts all anchor href attributes and writes them line-by-line in the file with the same name + “.links”. The sources are here.
I wrote an implementation of the Arc Challenge in Ruby using my web-framework Wee. The sourcecode is available here. I haven’t used continuations, because I think I broke continuation support in some of the later versions of Wee (just because I haven’t used continuations in my Wee apps). Once Rubinius gets (even) more mature, and my spare time allows, I’d like to get back to some ideas of Wee and make use of continuations.
The following sentence in the article Garbage Collection is Why Ruby on Rails is Slow: Patches to Improve Performance 5x; Memory Profiling made me shudder:
“Complex Rails request can allocate hundreds or even thousands of megabytes of memory, making GC runs dozens of times.”
WTF?! Hundreds or even thousands of megabytes for a single (but complex) Rails request? Then there is seriously something wrong!
In this post on Artima, Bruce Ekkel writes about his bad experience with web-sites that mix HTML and AJAX components. Especially some travel-sites that use this technology behave badly on other browsers than Internet Explorer, he says.
From my own experience I fully agree with him in this point. When I wanted to fly to Greece earlier this year I noticed that the GermanWings web site is not usable on any browser other than Internet Explorer because it displays an advertisement banner in front of the “choose your destination” dialog in the wrong moment. I guess thats clearly a CSS problem.
I disagree that this has anything to do with AJAX. At first, he mixes up AJAX with Javascript. AJAX is not Javascript! Most pages use Javascript for displaying a “date selector” dialog, and as he talks about new technologies, Javascript is clearly not one of them. Javascript is actually a damn old technology and date selectors exist for a very long time, so nothing fancy here. I think that especially for the user there is nothing easier to use than a Javascript-based date selector (unless you take technologies like Seaside or Wee into account ;-).
After a few hours of hacking, I implemented and refined the ideas first mentioned in my article HTML in Ruby and released XML in Ruby as you can read here.
For my blogging software I am using Tenjin as template engine. I also tried Kwartz which is also very nice. With Tenjin you embed Ruby into HTML code, in the same way as you’d do in Erb or PHP. This turns out to get pretty ugly. Kwartz completely separates HTML from presentation logic. It is a nice concept, but the presentation logic file can get quite complex for just simple things.
So why not embed HTML in Ruby?
# Example for embedding HTML in Ruby :)
class MyView
def render(posts)
for post in posts
render_post(post)
end
end
def render_post(post)
#<div id="${post.id}">
if post.abstract?
#<p>${post.abstract}</p>
else
#<p>${post.body}</p>
end
#</div>
end
end
I think you got the idea, right? I just misuse comments to embed HTML in Ruby. This is very easy to parse and looks quite nice in an editor and it should be easy to tell vim to colorize the embedded HTML correctly. Note that everything that starts with “#<” would be embedded HTML so that you’d still be able to use regular Ruby comments.
Nginx (aka. “EngineX”) is a high-performance web server similar to Lighttpd (aka. “Lighty”).
I migrated to Lighty from Apache long ago. Now it was time to migrate from Lighty to Nginx. Why? Well, I prefer it’s configuration over that of Lighty. It’s more intuitive. I just like it a lot!
The migration went seamless. And the configuration is now much more readable.
While working on pay4co.de (soon to be released), I saw the need for Captchas. Because I am lazy I found reCAPTCHA, which provides a service that you can include in your page. Not only that, it also helps to digitalize scanned in books, as the words the user has to type in come directly from scanned-in books. Here is the API Documentation.
A Capchta decoder for various captchas in use you can find here.
I've been using Kwartz lately and it's great! The good thing is that you can view your templates in your browser without the need to preprocess them first. Of course Kwartz only works for HTML or XML templates, but that's fine with me.
I'm currently extracting all German persons that are listed in Wikipedia. For this I used WWW::Mechanize, a library that I initially wrote, but now is maintained and extended by someone else. After fetching and parsing around 100 pages, the process took around 500 MB and pipes began to fail and stuff like that :). Mechanize keeps a history of all pages, so it's better to call history.clear from time to time, or maybe there is even an option to set the size of history. I found out about this after switching to using Hpricot directly (it is used by Mechanize anyway). So now memory stays constant with a maximum of around 30 MB.
The Hpricot HTML parser and XPath/CSS implementation is just - GREAT! So easy to use, so powerful, and I think it's also fast! Thanks _why!
At first I tried scrubyt!, but I never got it running. And after reading the documentation, I was quite confused. I think it tries to do too much and too much magic behind the scenes.
I'm not yet sure how I will name my library which is similar to Google Web Toolkit and is based on my RubyJS. Yesterday, I implemented some DOM related stuff in under two hours. GWT uses class methods of the DOM class everywhere, e.g.:
DOM.setElementAttribute(Element elem, String attr, String value)
I don't follow this approach, because I think it's more elegant to move them into an Element class and make it an instance method instead:
el = Element.createDiv
el.setAttribute("a", "b")
The same applies to Events. This allows me to write for example:
Element['root'].appendChild(
Element.createDiv.setInnerHTML('abc').
setAttribute('title', 'tooltip'))
instead of:
root = DOM.getElementById('root')
elem = DOM.createDiv
DOM.setInnerHTML(elem, 'abc')
DOM.setElementAttribute(elem, 'title', 'tooltip')
DOM:appendChild(root, elem)
which looks much less like OO in my hence opinion.
YSlow analyses your web page and tells you how and where you could improve it to make it faster (e.g. load times).
Google Gears is a browser extension that embeds an SQLite database into a browser to enable web application to even run when there is no internet connectivity. I think this is a great idea, once it (would) come by default with every web-browser.
I'd like to think about alternatives. I think the easiest alternative is to have a local web-server running on your system, which serves the client-side web-application even if there is no internet-connectivity. This of course requires parts (or the whole) of the server side to run under this local web-server. And thats my idea. Imagine you have your photos stored on Flickr. Do you trust them? I feel better when I have my photos at least backed up on my local system. And it's also much faster to show other people your photos once they are cached locally. So the idea is to have "two" servers. One locally and one globally. They synchronize each other, so that you can fully work offline without any need to change one line of the client side code. This of course means that you better open-source your "application", or you encrypt it so that no one can decipher it, if you prefer to protect your investments.
Okay, I did a simple performance benchmark on my laptop.
My 31 lines Erlang webserver shown below using the undocumented option {packet, http} as described here is very fast!
-module(http).
-export([start/0]).
start() ->
{ok, LSock} =
gen_tcp:listen(8081, [binary, {packet, http}, {reuseaddr, true}, {active, false}, {backlog, 30}]),
accept(LSock).
accept(LSock) ->
{ok, Sock} = gen_tcp:accept(LSock),
Pid = spawn(?MODULE, request(Sock)),
accept(LSock).
request(Sock) ->
{ok, {http_request, Method, Path, Version}} = gen_tcp:recv(Sock, 0),
headers(Sock).
headers(Sock) ->
case gen_tcp:recv(Sock, 0) of
{ok, {http_header, _, _, _, _}} -> headers(Sock);
{error, {http_error, _}} -> headers(Sock);
{ok, http_eoh} -> body(Sock)
end.
body(Sock) ->
gen_tcp:send(Sock, [<<"HTTP/1.1 200 OK\r\n">>,
["Connection", ": ", "Keep-Alive", "\r\n",
"Content-Length: 7\r\n",
"Content-Type: text/plain\r\n"],
<<"\r\n">>, <<"1234567">>]),
request(Sock).
Save this code in a file http.erl. Then start up Erlang "erl" and type "c(http)." followed by "http:start().". After that, you can run apache-bench on it:
ab -n 50000 -c 100 -k http://127.0.0.1:8081/test
The result on my very old laptop is 3600 requests/sec.
Now the same with Mongrel:
require 'rubygems'
require 'mongrel'
class TestHandler < Mongrel::HttpHandler
def process(req, resp)
resp.start(200) do |head, out|
head["Content-Length"] = "7"
out.write("1234567")
end
end
end
Mongrel::Configurator.new :host => '127.0.0.1' do
listener :port => 8081 do
uri "/test", :handler => TestHandler.new
run
join
end
end
The same benchmark with Mongrel gives only around 500 requests/sec. Of course Mongrel does a little more than the simple Erlang web server shown above. And I think Mongrel doesn't employ connection keep alive. But if you see that lighttpd serves only at 1050 requests/sec, then it's impressive what the Erlang web server can handle. Okay, lighttpd has to load the file from disk (7 bytes), that's much less efficient than serving directly from memory.
Read this. Having discovered it today, I have to say, I really like it :)
- Make session-id cookies available to both Wee::Request and Wee::PagelessRequest, controlled by an application-wide setting.
- Finish the implementation of pretty-URLs (this rulez!). Thanks to Joao Pedrosa who is working on this.
- Implement a FilterDecoration and it’s subclass AuthorizationDecoration (that’s very easy, ~20 lines of code).
- Implement a Seaside-like root_for traversal, which allows each component in the tree to add stuff inside the <head> tag. This is triggered by the root-component from it’s render method.
- Refactor the Wee::Session class and it’s subclasses.
- Implement further adaptors, most useful would be a FastCGI adaptor. This would need to factor out the WEBrick dependent parts.
- Implement ResouceHandlers, which make it possible to generate images (or other external resources) from within your components.
- Use it in your projects and write tutorials ;-)
Disclaimer: This is my understanding of how Rails work. I might be wrong.
Rails
Rails model is very simple compared to that of Wee. The controller class and the method to invoke is extracted directly from the URL. For example:
/blog/show
would invoke BlogController#show. Now imaging the following URL:
/blog/show/4
This would invoke BlogController#show with @params = {‘id’ => 4}. I don’t know how Rails knows that 4 maps to the ‘id’ key, but that’s another story.
So you have these parts of a URL:
/controller/action/arguments
In Rails, there’s also no (conceptual) distinction between performing an action and rendering, as is the case in Wee. Both are inseparable from each other (in Rails).
Another difference to Wee is, that the controller classes are stateless, and instances of it might be reused (pooling). On the other hand this implicates that you can’t store information inside a controller instance across requests.
Wee
In Wee, we differenciate between an action phase, where callbacks are invoked and as such possibly modify the state of components, and a render phase, which ideally is side-effect free and whose sole purpose is to render the component to HTML (or whatever format you like).
Imagine you look at a simple Wee application which displays an HTML anchor tag. If you click on that anchor, you’ll trigger an action phase, which eventually will find the registered callback associated with the anchor you have clicked on and invokes it. At the end of the action phase, Wee will forward you to a new (automatically generated) URL, which when requested by your browser, will trigger the render phase. This in turn renders the whole component tree. "Why redirect", you might ask. Well, this simply avoids that the same callback will be invoked again if you hit your browsers reload button. Whenever you hit on reload, this will only trigger a render phase event.
Additionally, at the end of each action phase, a snapshot of the component tree is taken, so that you can go back to older states (called back-tracking). The information about which state of the component tree we refer to, is stored as page_id inside the URL. This page_id increases whenever an action phase is performed.
So, an URL in Wee basically consists of the following parts:
session_id page_id [callback_id]
where callback_id is optional. If it is given, the URL triggers an action phase. Otherwise, a render phase.
Compared to Rails controllers, Wee’s components are composites, meaning that they may contain sub-components which itself might contain sub-components and so on. And in Wee there’s only ever one root component, whereas in Rails there are usually multiple controllers. All this makes it nearly impossible to have REST-like URLs in Wee. Also due to the reason that a sub-component cannot be rendered on it’s own, whereas a controller in Rails builds a whole page. So it does not make sense to have URLs in Wee like:
/1.2.3
# == @root_component.children[1].children[2].children[3]
Marrying Wee and Rails aproaches
The concept of multiple top-level controllers can be easily added onto Wee by using the following RootComponent class:
# NOTE: not fully functional code!
class RootComponent < Wee::Component
def initialize
super()
@controllers = {
'blog' => add_child(BlogComponent.new),
'list' => add_child(ListComponent.new)
}
end
def render
controller = # extract information from URL
r.render @controllers[controller]
end
end
This RootComponent merely acts as a dispatcher. It looks at the URL and extracts the desired controller out of it and then forwards to it.
Likewise, we could map the Rails action part (the "show" in "/blog/show"), to invoke render_xxx (e.g. render_show) of the controller-component (I use this term now, to distinguish it from a "regular" Wee component and to mark the similarity with a Rails controller):
def render
controller = # extract information from URL
action = # extract Rails action part from URL
component = @controllers[controller]
component.with_renderer {
component.send("render_" + action)
}
end
And even further, we could also extract the additional arguments of the URL for use inside the called render method. We could use a custom parse_arguments method here in each controller-component.
Now lets look at a simple example:
class BlogController < Wee::Component
def parse_arguments(str)
# for /blog/show/5, str would be "5"
@params = ... # e.g. {'id' => 5}
end
def render_show
entry = BlogEntry.get(@params['id'])
# render it
end
def render_list
BlogEntry.find_all do |entry|
# render it
end
end
end
What else would be needed is to tell which controller/action pair should be used. This could for example be specified each time when generating an anchor or form tag:
r.anchor.controller('blog/show/5').callback { .... }.with('show')
Note that this would first invoke the callback and then render ‘blog/show/5’. Of course you could ommit in this case the callback. But specifying the controller/action pair each time is tedious, as this would have to be done in each sub-component, too, which completely breaks the concept of a component. So the second approach is much better:
r.anchor.callback {
... do something
request.controller = 'blog/show/5'
}
where the request.controller setting is carried inside the URL as long as you assign a new value to it.
Note that you can still use sub-components with this approach, but they would no longer be subject of REST-like URLs.
Some days ago, I wrote an OgScaffolder component. It’s damn easy to use. Just pass an Og domain class to OgScaffolder.new, and what you get is a regular Wee component. Look at the screenshot below.
Below I’d like to show some Nemo screenshots, to show it’s potential. Note that the whole generic editor component (the first screenshot) is less than 200 lines very clean Ruby code! Thanks Kevin ;-)
It depends on narf-lib (>= 0.6.3), more exactly, it’s version of htmltools, and requires ruby-current (due to limitations in the net/http library of 1.8.2). If you want to try it out with Ruby 1.8.2, just get the net/*.rb files from ruby-current and use these instead.
Features
- Cookies (very preliminary, no expire etc.)
- automatic redirect
- Forms, Links
Be aware that it’s just a few-hour hack, to get my project done.
It’s still very generic and misses lots of usability methods. For example, I use find all the time, or have to pass the link that should be clicked to method click.
Example
Here’s a simple example, that logs into your rubyforge.org acount (pass your username and password as command-line argument).
require 'mechanize'
agent = WWW::Mechanize.new {|a| a.log = Logger.new(STDERR) }
page = agent.get('http://rubyforge.org/')
link = page.links.find {|l| l.node.text =~ /Log In/ }
page = agent.click(link)
form = page.forms[1]
form.fields.find {|f| f.name == 'form_loginname'}.value = ARGV[0]
form.fields.find {|f| f.name == 'form_pw'}.value = ARGV[1]
page = agent.submit(form, form.buttons.first)
puts page.body
Get it!
The screenshot shows the edit mode. If you click on ‘Back’, then the left pane is hidden and you can proceed with presenting. If you click with the mouse on the title, the next overlay (or slide, if we’re on the last overlay) is shown. I want to add keyboard navigation, too, but I couldn’t get the JavaScript working. Note that the whole application is in no way JavaScript-driven, I only need JavaScript to trigger an "open(url)" call, so that Wee will display a new page. It would work completely without JavaScript if I would display anchor links for "next overlay" and so on.
Features
- The presentations file format uses Ruby’s documentation markup (RDoc), which is a very easy markup language (nearly plain text). So you even don’t need the web-based editor to create the presentation, you could also use your favourit editor (vim of course ;-)
- It’s easy to display colorized source code. Just insert:
!!colorize:ruby def this_method end - You can start applications from within the presentation by clicking on a
link. Note that the application is started on the server-side, which is
intended, as you’ll run the presentation maker application from your
very own machine. Example (the execute button in the screenshot):
!!exec:hidden cd $HOME/Dev/svn/public/ForSys && ruby test_lout.rb - You can create Postscript and PDF output for printing. A Table of Content is included, and in the PDF version all links are clickable (of course all images are included, too).
- Of course you can use your own style-sheet.
Try it!
viewcvs.ntecs.de/PresentationMaker/
And don’t forget to read the README file.
www.ntecs.de/viewcvs/viewcvs/*checkout*/Wee/trunk/doc/rdoc/index.html
Let’s look at some scenarios. I’ll call the object to register "obj" and it’s snapshot "snap". I further assume that "snap" references "obj".
Scenario 1 (Good)
- We register "obj" for backtracking.
- "obj" goes out of scope -> it gets garbage collected.
- We take a snapshot. No snapshot of "obj" is taken.
Here, no problem.
Scenario 2 (Worse)
- We register "obj" for backtracking.
- We take a snapshot "snap".
- Snapshot "snap" goes out of scope.
- Now "obj" could go out of scope, too.
Note that as long as a snapshot of "obj" exists, "obj" cannot go out of scope.
Scenario 3 (Worst)
- We register "obj" for backtracking.
- We take a snapshot "snap".
- Before "snap" goes out of scope, we take another snapshot. This new snapshot also includes "obj".
As long as at least one snapshot exists that was taken after the initial "snap" snapshot, "obj" will not be garbage collected. It’s included in every further snapshot.
This might happen in Wee as we store a fixed number of snapshots (there’s always one available). So you have to take care to not introduce a reference back to the registered object from the snapshot.
Solutions
- Whenever possible, avoid to register objects in dynamically created components (e.g. MessageBox).
- The snapshot of a registered object which is taken by calling method take_snapshot of the objects’s class should never contain a reference back to the referenced object. If this happens, you’ll introduce memory-leaks (for the duration of the session).
Last evening, we then build the calculator shown in the picture below. It’s a portable web application using Ruby and my new Wee framework. I spent most time on generating the diagram (it’s done using gnuplot). The rest was straightforward, once we had the underlying calculations working.
The total code size is 360 lines of code (including HTML), of which 110 are used for the calculation and for generating the diagram. Lots of lines of code are currently spent in conversion and validation routines (german number format).
You can try it out online here: ntecs.de/wee/ehf/
Blocks
In the rendering-phase, you usually define some actions and input elements. This is currently done this way:
def inc; @cnt += 1 end
def dec; @cnt -= 1 end
def render_content_on(r)
r.anchor.action(:inc).with('++')
r.anchor.action(:dec).with('--')
end
While with blocks, you can write:
def render_content_on(r)
r.anchor.action { @cnt += 1 }.with('++')
r.anchor.action { @cnt -= 1 }.with('--')
end
When it comes to storing a session on disk, we simply mark those pages for which a block handler was specified, as "needing another render-pass before being able to respond to an action". So, in the rare case of needing to shutdown the computer, we don’t loose any information. It’s just that the user has to reload the page.
Continuations
We can rebuild block handlers, by executing their definition (render-phase). But that does not apply to continations. If you serialize a session with active continuations, Wee will replace the continations with marker objects, so that we can later detect that there was an active contination going on. If this happens, Wee will notify the user about this situation and will return to the calling component and call it’s method fallback_from_cc. If you want, you can still use old-style call behaviour by passing the name of the return method. Then you won’t loose any information.
Another idea is to use DragonFly's checkpointing support. This would be a 100% solution for the whole problem.
This is an average of 150 requests per second or if we don’t count the redirects, which make the half of the requests, this is still an impressive performance of 75 whole page views per second (on a 1300 MHz Centrino laptop, running KDE with lot’s of open applications). Of course this time, the web application was a very simple one, it showed only a chain of dialog boxes.
The rest of the day I fighted against some memory issues. To detect them, I was in the need for a simple stress testing application. Thus, I patched WebUnit, to support redirects and <br/> tags, then wrote a memory observer with GNUplot output and finally the stress tester.
I run the tests with all recent versions of Ruby, then I realized that the leak must be in my program and not in Ruby. More or less quickly I discovered that I had a reference back to the continuation in the called component, which resulted in the continuation not being freed, as the continuation has itself a reference to the component on the procedure stack. Assigning nil fixed the bug.
Below is the memory usage of Wee with continuations running under ruby-current as of today. I stressed it with 50 processes, i.e. 50 active sessions (= 50 Ruby threads), and a total of 100000 requests (in 10 minutes). It stayed below 22MB, that’s 500kB per session. It still slightly increases over the time, not sure whether this is GC related or not (fragmentation?). I’ll have to stress it over night ;-)
Backtracking decorations is now possible, too, but you still have to do this on your own. This is useful, so that "call/answer" calls can be undone correctly.
Decorations
A Delegate decoration is used to implement the call/answer mechanism. I’ll explain this by demonstrating a little example:
def render_content_on(r)
r.anchor.action(:confirm_quit).with do
r.text('Quit')
end
end
def confirm_quit
call MessageBox.new('Do you really want to quit?'), :quit
end
def quit(res)
exit if res
end
If you click on the anchor named Quit, method confirm_quit is called. This then replaces the calling component with a message box (or at least, only the message box is shown and can handle actions, not the calling component), with two buttons OK and Cancel. If you now click on one of them, the calling component is restored and it’s method quit is invoked with the return code of the message box. How the message box can be implemented is shown below:
class MessageBox < Wee::Component
def initialize(msg)
@msg = msg
end
def ok
answer true
end
def cancel
answer false
end
def render_content_on(r)
r.text @msg
r.form do
r.submit_button.value('OK').action(:ok)
r.space
r.submit_button.value('Cancel').action(:cancel)
end
end
end
In further versions of Wee, it will be possible to register an action and specify some default arguments. This would make the methods ok and cancel superfluous:
r.submit_button.value('OK').action(:answer, true)
r.space
r.submit_button.value('Cancel').action(:answer, false)
Uhm, 12 minutes later… it’s working ;-) Found by the way two other bugs.
So how is this call/answer mechanism working? Well, it really is only half a dozen lines of code:
class Wee::Component
attr_accessor :caller # who is calling us?
# call another component
def call(component, return_method=nil)
component.caller = [self, return_method]
add_decoration(Wee::Delegate.new(component))
end
# return from a call
def answer(*args)
component, return_method = self.caller
component.remove_first_decoration
component.send(return_method, *args) if return_method
end
end
Get it! Try it!
So now, get Wee and try it out: viewcvs.ntecs.de/Wee/
Or play with it here.
val.dup rescue val
into:
case val
when Fixnum, nil, true, false
val
else
val.dup rescue val
end
The second is up to 14 times faster for objects that can’t be cloned (Fixnum, nil, true, false). This results in an increase from 97 req/sec to 160 req/sec in the action-phase of Wee (for the example described in WeeFramework). Next, I will do component render caching (for leaf components).
The core is now around 600 lines of code. The HTML related libraries put another 500 lines of code.
The trickiest thing I implemented was the StateRegistry, where you can register objects for being backtracked, and where you can take snapshots of all registered objects. All this is done with extensive use of weak-references and finalizers. The serialization part of the StateRegistry was tricky too, as you can’t simply serialize object-ids.
Features
Fully serializable
If you shut down the application, all state gets stored to disk. You can then reload the whole application from disk (possibly on another machine) with exactly the same state where you left off. Of course you could also store the sessions in a database table. Should be simple to implement any kind of load-balancing, as each session is completely independent, so that you could simply transfer sessions to another machine.
Backtracking
If you want, you can make the back-button of your browser work correctly together with your web-application. Imagine you have a simple counter application, which shows the current count and two links inc and dec with which you can increase or decrease the current count. Starting with an inital count of 0, you increase the counter up to 8, then click three times the back button of your browser (now displays 5) and finally decrease by one, then the counter really shows the expected 4, instead of 7 (as clicking the back button does usually not send a HTTP request, and the last state of your application was 8).
Only individual objects are backtracked, those that are registered, not the whole component. That’s the easiest (from an application programmers perspective) and most flexible way. And it’s fast and uses little memory.
You can decide yourself whether you want infinite backtracking or only backtracking up to n pages, with whatever replacement strategy you want, least recently used (LRU), least frequently used (LFU) etc.
Acceptable Performance and Memory Usage
Memory consumption stayed constant in all of my tests (with Ruby 1.8.x, not with 1.9!) by around 7 MB. On my Centrino 1300 MHz laptop, I can currently render 86 pages per second and can handle 97 action requests per second. That means it can handle 45 complete page views per second. The example consist of 20 counter components, with backtracking each of them, WEBrick as webserver (100% pure Ruby).
Reusable Components
Wee components are like widgets in a GUI. Once written, you can use them everywhere. They are completely independent and do not interfere with other components. Components encapsulate state, a view and actions. Of course you can use an external model or use templates for rendering.
Programmatic HTML Generation
You don’t have to leave Ruby to generate the HTML code. This is especially useful for highly dynamic components. But of course, you could also use templates.
Example
Below is the screenshot and sourcecode of a simple editable counter component that is encapsulated 10 times in the main component. You can try it out live here (it’s quite slow as it gets proxied through Apache as I’m too lazy to open my firewall).
require 'wee'
class Counter < Wee::Component
def initialize(cnt)
@cnt = cnt
@show_edit_field = false
session.register_object_for_backtracking(self)
end
def dec
@cnt -= 1
end
def inc
@cnt += 1
end
def render_content_on(r)
r.form.action(:submit).with do
r.anchor.action(:dec).with("--")
r.space
if @show_edit_field
r.text_input.assign(:cnt=).value(@cnt).size(6)
else
r.anchor.action(:submit).with(@cnt)
end
r.space
r.anchor.action(:inc).with("++")
end
end
def submit
@show_edit_field = !@show_edit_field
end
def cnt
@cnt
end
def cnt=(val)
@cnt = val.to_i if val =~ /^\d+$/
end
end
class Main < Wee::Component
def initialize
@counters = (1..10).map {|i| Counter.new(i)}
children.push(*@counters)
end
def render_content_on(r)
r.page.title("Counter Test").with do
@counters.each { |cnt| r.render(cnt) }
end
end
end
Then Armin asked me if I would like to do a presentation about Ruby and the Web at EuRuKo04, but I couldn’t decide on a single powerful framework which rulez the world (in my eyes).
Why not Rails, IOWA or Borges?
I never got behind Rails and why it should be so good (luckily I’m not the only one). IOWA instead I know for serveral years and like it’s underlying model. What I don’t like in IOWA is listed below (this should not seen as criticism):
- The build-in templating stuff that you can’t get around,
- that IOWA components are in some ways different than pure Ruby classes,
- it’s code layout (yeah, in this way I’m an aesthet),
- the use of class-variables and too much magic like "inherited" or import instead of require,
- but most importantly, that I don’t understand all details of the code
The Borges framework is very nice and powerful, but has had some severe problems with memory leaks in the past, probably due to using continuations (I am not sure whether they were able to fix all these issues). Borges is a port of Seaside 2 from Smalltalk (IIRC, IOWA was a port of Seaside 1). Currently, there seems to be no further activity in improving Borges and make in en par with Seaside. Furthermore I heard on a mailing list that it is very slow due to continations or whatever else, and as such not suitable for larger applications. And you can’t migrate sessions between processes, at least not in Ruby, as Continations are not serializable.
My goals
In short, I wanted something that looks like Borges, but with the underlying model of IOWA. And of course which is developed in Ruby and for Ruby from the first second on.
I pretty exactly knew how a component in my WebFramework (put in better name) should look like. See the example section below.
So what actually were my goals?
- Web applications and components should look very concise and clean. Components should in no way look different than usual classes in Ruby. You can use instance variables as you would use them in ordinary Ruby classes. Initialization is no magic and works as usual.
- Make it easy to develop reusable components.
- No configuration files or special setup required. Works out of the box.
- To be able to migrate sessions and serialize them and the whole components tree to a file.
- Be able to not just use this framework for HTML content generation, but also for example for emails. Imagine Python’s mailman. This could be implemented "easily" with this framework by using long-term sessions and a special email module that extracts the session and action id out of the email. Then, the only difference to the mailman web-module would be the content generation (plain text for email vs. html for web), the actions would stay the same. This is why I did not have chosen continuations for my framework, as they are not good for long-term sessions, because you loose them if you have to shutdown your server (and sessions for email usually endure several days).
- Have a nice framework that I fully understand.
Example
Below, I implement a simple counter component and the main component that displays ten counters.
require 'web'
class MainPage < Web::Component
def setup
@body = "<html><body><p>hello world</p>%s</html>"
@counter_list = (0..10).map {Counter.new(self)}
end
def render
@body % (@counter_list.map {|c| c.render}.join("<br>"))
end
end
class Counter < Web::Component
def setup
@counter = 0
end
def render
%{ <a href="#{ action_url(:dec) }">--</a> #{ @counter }
<a href="#{ action_url(:inc) }">++</a> }
end
def inc
@counter += 1
end
def dec
@counter -= 1
end
end
Using templates
Of course you could use templates, too. I quickly implemented this in a few lines of very tricky Ruby code.
require 'rdoc/template'
module Templating
def template
file = caller.first.split(':').first
file = file[0..(-1-File.extname(file).size)] + ".tmpl"
tmpl = TemplatePage.new(File.read(file))
proc {|values| tmpl.write_html_on('', values) }
end
end
Now let’s look at how the templated version of the counter component would look like. You don’t even have to specify the name of the template.
class TemplatedCounter < Counter
include Templating
def render; template[
'dec' => action_url(:dec),
'inc' => action_url(:inc),
'counter' => @counter.to_s]
end
end
The template should be in the same directory as the component. Of course only if you use my approach shown here, which I like very much, as this allows to put each component in it’s own directory, together with it’s template, and you don’t need to setup template paths etc.
<a href="%dec%">--</a> %counter% <a href="%inc%">++</a>
Starting the application
All examples above just implemented the components. To start the application using Ruby’s WEBrick webserver, you have to write:
require 'web/webrick'
Web::Application.new {|app|
app.name = 'Counter'
app.session_factory {
Web::Session.new(app) { |sess| sess.root_component = MainPage.new }
}
}.start









