The Twitter web interface allows you to take a look at older tweets in bunches of about a twenty each. And every time you want to fetch a new batch you have to click on the large “more” button. Which in turn often brings up the message features in this blog post’s title.

I’m annoyed as hell by this of course, but it also gives me the opportunity for a little analysis that will end in the conclusion: Twitter will fail (unless they take some spoons of Tech Clue and change radically, that is).

What’s behind the little error message? You can’t really say for sure what happened, but since these errors occur pretty randomly it’s probably just that their servers are very busy right now.

Plus Twitter is notoriously known for having problems with scalability. For roughly the last two years, in fact. If you ask me they must be pretty clueless about scalability issues. To my knowledge their last action to solve this was to switch to another programming language (Scala). This approach seems to be motivated more by the name of the language than the actual benefit derived from switching to it. I don’t know if they have made the switch yet or whether they changed their strategy, but chances are their scalability problems won’t be solved at all with this.

The only thing that could cure their scalability worries would be a proper protocol for distributed twittering. There are suitable and proven distributed protocols like XMPP. You know, that’s the thing the Jabber instant messaging network is built on. And Google Wave too. This is quite a clue on the ability of Wave to supersede Twitter. Instead the latter ones are wasting their time with fruitless efforts. Come on!

This is made worse by the sheer simplicity of their application. I mean, in one of my projects I’m the sole developer of a complex game application. It’s understandable that I can’t be all things for all people, and scalability is just one of many problems. But what the hell are they doing all day at Twitter? They have 29 employees right now (source: TechCrunch)! Are they all busy keeping the list of top tweets fresh, or making Excel diagrams to please their VCs?

What’s your opinion on Twitter, scalability and the future of micro-blogging?

It is designed to make Agile web application development as
effective and simple as possible.

WHY YET ANOTHER WEB FRAMEWORK?

This is not your ordinary run-of-the-mill web framework in PHP, Python or Ruby.

Weblocks uses powerful Lisp features like multiple dispatch, the
metaobject protocol, lexical closures, keyword arguments, and macros
to build abstractions that make web development easy, intuitive, and
free of boilerplate. In addition, control flow is easily expressed
using continuations.

Things that are hard or mundane in other frameworks become easy and
fun in Weblocks. A fine example of this are Weblocks’ AJAX capabilities
which relieves you from writing Javascript in a lot of situations.

AUDIENCE

People who want to get their real-life web programming done
as effectively as possible.

Weblocks is not only targeted at old hands but also at
newcomers to Lisp and Lisp web programming.

It offers a helpful community and code that prevents you
from shooting yourself too easily in the foot.

IS IT USABLE? CAN I SEE SOME DEMOS?

Weblocks is well-tested and has proven its worth in daily usage.
It is used by a community of developers all over the world.

Public applications running Weblocks include

• LAMsight
• Aula Polska
• Thanandar

CHANGES IN 0.8.3

• Greatly enhanced performance in the request handler
• Support for request timeouts
• Automatic bundling/versioning of static files
• Rudimentary profiling
• Fundamentally overhauled nav system
• CLSQL demo fixed
• Updated Simple Blog example
• New HTML error handler
• More flexible debugger configuration with additional restarts
• The usual assortment of fixes and contrib/ additions

MORE INFORMATION

Platforms

Well-tested on SBCL and Clozure CL.
Partially tested on CMUCL, Lispworks, AllegroCL, and OpenMCL.

Official site (with detailed installation guide)

http://weblocks.viridian-project.de/

Demo

http://weblocks.viridian-project.de/weblocks-demo

CONTRIBUTORS

This release has been made possible by Nandan Bagchee, Benjamin Collins,
Stephen Compall, Chris Hallwright, Jan Rychter and yours truly.

Fortunately the PEAR project has a set of excellent (well, as close to excellency as PHP lets you get…) libraries for all sorts of purposes.

The only thing with them is that they have problems managing their releases. The main site of the popular HTML Quickform library sports a fat warning sign, claiming that this package had been superseded by the aptly named HTML QuickForm 2 package.

That’s funny, because most if not all other packages depending on QuickForm are still built for the old version. Even more funny is what a vanilla install of the PEAR package tool will throws at anyone attempting to install QuickForm2:

% sudo pear install HTML_QuickForm2
Failed to download pear/HTML_QuickForm2 within preferred state "stable", latest release is version 0.2.0, stability "alpha", use "channel://pear.php.net/HTML_QuickForm2-0.2.0" to install
Cannot initialize 'channel://pear.php.net/HTML_QuickForm2', invalid or missing package file
Package "channel://pear.php.net/HTML_QuickForm2" is not valid
install failed

Now take into account that this situation has been in place for at least one year. Not really a role model for a sane software development release process.

Evidence B: Structures_Datagrid and a bunch of other packages are also marked “beta” but have been used by the majority (I daresay) of developers without any problems over the course of the last dekamonths.

What gives, PEAR?

Very useful for testing local mail emitting routines.

Usage: download, extract, quickly edit the configuration files and execute bin/mail.sh.

Thanks, Eric!

urlcheck is lightweight, concurrent url checker. It uses minimal resources, and is multicore-capable.

In contrast to popular link checkers, it uses few resources, will readily take advantage of multiple cores, and is easily killable.

Works great and is fast as lightning. Plus, it’s written in Haskell.

Arch Linux users get it from AUR, all others from their distribution or directly from source.

SSI doesn’t really help because its syntax is clumsy and the files are compiled again on each request. It also needs to be enabled on the target host.

A solution that compiles standard HTML from templates is best for performance and portability. It’s possible to use on of the many HTML template toolkits out there, but why bother? All I want is a standardized header and footer and a page title varying with each HTML file.

So let’s do just that:

#!/bin/bash
set -e
 
OUTDIR=./build
 
mkdir -p "$OUTDIR"
 
# wrap header and footer, set <title> from first line of body
for f in {page1,page2,page3}.html; do
  TITLE=$(echo $(head -n1 $f) | perl -p -e 's/^TITLE: *(.*?)$/$1/')
 
  (sed "s|TITLE|$TITLE|" header.html ;
   sed '1d' $f;
   cat footer.html) > "$OUTDIR/$f"
done
 
# auxiliary files
for f in logo.png default.css; do
  cp $f "$OUTDIR/$f"
done
</title>

The files that get wrapped into header and footer need to set their title on the first line:

TITLE: my page title
<p>So this is it...</p>

If you don’t like Perl, you can also whip up a sed or awk script in its place.

(in-package :weblocks)
 
(export '(table-composite render-widget-body))
 
(defwidget table-composite (composite)
  ((cols :type integer :accessor cols :initarg :cols :initform 0))
  (:documentation "Renders a set of widgets in table layout."))
 
(defmethod render-widget-body ((widget table-composite) &rest args)
  (let* ((widgets (composite-widgets widget))
         (num-widgets (list-length widgets))
         (cols (cols widget))
         (rows (ceiling (/ num-widgets cols))))
    (with-html
      (:table :rows rows :cols cols
        (loop for r from 0 below rows
              do (htm (:tr
              (loop for c from 0 below cols
                    do (let ((child (nth (+ c (* r cols)) widgets)))
                         (htm (:td (when child (render-widget child)))))))))))))
 
;;; EXAMPLE CODE:
(make-instance 'table-composite :cols 3
  :widgets (loop for i from 1 to 10
                 collect (write-to-string i)))

Yeah, I know a DOLIST would do the same as the LOOP here, but I really like LOOP.

This should be easily adaptable for generic HTML rendering.

What particularly struck me was his comparison of HTML tables with Lisp’s untyped lists.

First, Paul talks about the evolution of his opinion on untyped lists:

“I went through a stage, after I’d been programming in Lisp for 2 or 3 years, where I thought the old way of using lists to represent everything was just a hack. If you needed to represent points, surely it was better to declare a proper structure with x and y fields than to use a list of two numbers. Lists could contain anything. They might even have varying numbers of elements.

Over the years my appreciation for lists has increased. In exploratory programming, the fact that it’s unclear what a list represents is an advantage, because you yourself are unclear about what type of program you’re trying to write. The most important thing is not to constrain the evolution of your ideas. So the less you commit yourself in writing to what your data structures represent, the better.”

I’m sure most Lisp programmers will agree to this.

Now, what about HTML tables?

“Tables are the lists of html. The W3C doesn’t like you to use tables to do more than display tabular data because then it’s unclear what a table cell means. But this sort of ambiguity is not always an error. It might be an accurate reflection of the programmer’s state of mind. In exploratory programming, the programmer is by definition unsure what the program represents.”

And yes, CSS still can’t do more than three columns, and even for three I would have to look it up.

It’s obviously not really a CMS that one would install for a customer, but it’s a nice thing
for your typical software projects: a flock of small pages gathering around the most important thing,
namely the downloads.

Well done!

Here’s a widget for Weblocks that will create a widget from static HTML:

(in-package :weblocks)
 
(export '(static-html make-static-html-from-file
          with-widget-header render-widget-body))
 
(defwidget static-html (widget)
  ((html :type string :accessor html :initarg :html :initform ""))
  (:documentation "Represents a piece of static HTML body mark-up."))
 
(defmethod with-widget-header ((widget static-html) body-fn &rest args &key
                                                    prewidget-body-fn postwidget-body-fn &allow-other-keys)
    (apply body-fn widget args))
 
(defmethod render-widget-body ((widget static-html) &rest args)
  (format *weblocks-output-stream* "~A~_" (html widget)))
 
(defun make-static-html-from-file (file)
  "Create a static-html widget representing the mark-up in “file”."
  (with-open-file (input file :direction :input)
    (let ((data (make-string (file-length input))))
      (read-sequence data input)
      (make-instance 'static-html :html data))))

The Lisp paste is at http://paste.lisp.org/display/54023.