Space Station Lambda

As a precursor to 0.9 which will contain much desired features like AUR system upgrades we decided to release 0.8.3 which contains a bunch of bug fixes:

• Version comparison no longer fails on provider packages (#8)
• Reinstallation works properly again (#7)
• Trying to install or get pkgbuilds for non-existent packages is handled correctly (#5 and #6)
• Question the user in case of malformed pkgbuild dependencies (#12)
• Proxy support works correctly now (#15, reported by nitralime)
• update cache after package removal (proposed by Ralith)
• handle non-Unicode strings more gracefully (#9, reported by zajca)

Syncing up

Use the quickinstall script or sync your Paktahn repo to get it.

Comments(3)

Merry christmas everyone!

It must be christmas of course since Paktahn 0.8.2 is now out as promised! ;D

As it often happens in software development it’s a little later than originally expected, but there’s a lot of good stuff that has made it into this release.

Highlights

• fixed arch=(any) case (reported by magus)
• proper error reporting and restarts when AUR results cannot be fetched (Brit)
• Paktahn now remembers which PKGBUILD files it already presented for review (Brit)
• Paktahn now has proper customizepkg support for AUR packages and will automatically build packages with customizepkg definitions from source (Brit)
• support for just getting a pkgbuild (i.e. yaourt -G) with pak -G pkgnames (Brit)
• makepkg’s PKGDEST variable is detected and used correctly (reported by Stefan Husmann)
• AUR package dependencies are no longer installed explicitly (reported by bram85)
• Basic proxy support (no authentication) (Brit)
• Basic versioning support (Wei Hu, Leslie)

Also, we no longer depend on a custom version of SBCL!

Thanks to Jürgen Hötzel, Wei Hu and of course my colleague Brit Butler for their help with this release.

Syncing up

Use the quickinstall script or sync your Paktahn repo to get it.

Comments(2)

Here’s a one-liner that will install Paktahn.

It will add the Paktahn binary repository to /etc/pacman.conf and run Pacman to install Paktahn.

wget -qO- http://viridian-project.de/\~sky/paktahn/quickinstall.sh | sudo sh

You just need to paste this line into a terminal. Review the script here, it’s very simple:

http://viridian-project.de/~sky/paktahn/quickinstall.sh

Comments(4)

The next version of Paktahn is ready.

Just sync-install your Paktahn package; rc testers will have to remove and reinstall explicitly.

New users should refer to the original blog post for instructions.

Changes

Bug fixes

* fixed x86_64 package building/installation
* fixed case when $EDITOR also had args in it (e.g. “emacsclient -l”)
* throw an error when the user attempts to build an AUR package as root
* fixed –help and -h arguments (reported by Ralith)

New features

* package removal support
* support for provider packages
* support for reinstallation of packages
* ask the user whether he wants to add his arch if it’s not listed in the PKGBUILD.

Comments?

As usual please let me know of any bugs and your wishes.

Comments(2)

Hi folks,

Paktahn 0.8.1 is ready but has only been tested by me, so here’s a prerelease for you to run and help find bugs:

http://viridian-project.de/~sky/paktahn/rc/paktahn-0.8.1-1-i686.pkg.tar.gz

Changes:

* package removal support
* support for provider packages
* support reinstallation of packages
* throw an error when the user attempts to build an AUR package as root

You’ve already been very supportive with your feedback for 0.8! Thanks!

Comments(2)

[This post is geared towards ArchLinux users. It might confuse you if you're not part of that audience.]

yaourt, the popular Pacman wrapper, doesn’t seem to be maintained anymore and has quite some problems (e.g. UI nits, security issues, huge shell codebase). And people are starting to get tired of it because basic things stop working.

But since yaourt’s interface is so great I have decided to code up an alternative called Paktahn (after a ship type from Wing Commander).

But what can it do?

For now it supports yaourt’s two most helpful (IMHO) operation modes:

pak query

Search AUR and repositories for QUERY in packages names and descriptions and display the results. Ask for one or more packages to be installed.

pak -S package

Install PACKAGE from repositories or AUR.

Paktahn comes with a highly efficient caching system resulting in a speed that will blow your mind if you’re used to Pacman or yaourt.

I’ve been using the current version of Paktahn for weeks now and it has never failed me.

How to get it

Add one of the following sections to your pacman.conf:

# i686
[paktahn]
Server = http://viridian-project.de/~sky/paktahn/repo

# x86_64, package contributed by Gour
[paktahn]
Server = http://viridian-project.de/~sky/paktahn/repo/x86_64

There’s no x86_64 package yet since I cannot build them and no one else has contributed one yet. However you can build it from source: Paktahn on AUR. You need to install a patched version of SBCL before attempting to do so.

Development happens here: http://developer.berlios.de/projects/paktahn/

Discussion thread: http://bbs.archlinux.org/viewtopic.php?id=77674

Contributions, bug reports, comments and praise are all welcome, as are donations since I’m an independent software developer.

Comments(28)

If you’re a Twitter user then chances are you’ve seen this message. Here’s a rant based on it.

The Twitter web interface allows you to take a look at older tweets in bunches of about a twenty each. And every time you want to fetch a new batch you have to click on the large “more” button. Which in turn often brings up the message features in this blog post’s title.

I’m annoyed as hell by this of course, but it also gives me the opportunity for a little analysis that will end in the conclusion: Twitter will fail (unless they take some spoons of Tech Clue and change radically, that is).

What’s behind the little error message? You can’t really say for sure what happened, but since these errors occur pretty randomly it’s probably just that their servers are very busy right now.

Plus Twitter is notoriously known for having problems with scalability. For roughly the last two years, in fact. If you ask me they must be pretty clueless about scalability issues. To my knowledge their last action to solve this was to switch to another programming language (Scala). This approach seems to be motivated more by the name of the language than the actual benefit derived from switching to it. I don’t know if they have made the switch yet or whether they changed their strategy, but chances are their scalability problems won’t be solved at all with this.

The only thing that could cure their scalability worries would be a proper protocol for distributed twittering. There are suitable and proven distributed protocols like XMPP. You know, that’s the thing the Jabber instant messaging network is built on. And Google Wave too. This is quite a clue on the ability of Wave to supersede Twitter. Instead the latter ones are wasting their time with fruitless efforts. Come on!

This is made worse by the sheer simplicity of their application. I mean, in one of my projects I’m the sole developer of a complex game application. It’s understandable that I can’t be all things for all people, and scalability is just one of many problems. But what the hell are they doing all day at Twitter? They have 29 employees right now (source: TechCrunch)! Are they all busy keeping the list of top tweets fresh, or making Excel diagrams to please their VCs?

What’s your opinion on Twitter, scalability and the future of micro-blogging?

Comments(1)

The POSIX threads interface allows user applications to manage scheduling policies and priorities. Certain threads can be made to wait in favor of other threads (priority), and the order of execution for threads of equal priority can be specified (policy).

Below is a humble example interface to access this API.

(asdf:oos 'asdf:load-op 'cffi)
 
(defpackage :prio (:use :cl :cffi))
 
(in-package :prio)
 
;;; constants
;; CFFI-GROVEL offers a more reliable solution for this,
;; but let's keep it simple.
(defconstant +sched_other+ 0)
(defconstant +sched_fifo+ 1)
(defconstant +sched_rr+ 2)
 
 
;;; define the pthread library
(define-foreign-library libpthread
  (:unix "libpthread.so.0"))
 
(use-foreign-library libpthread)
 
 
;;; C structures and functions
(defcstruct sched_param
  (sched_priority :int))
 
(defcfun "pthread_getschedparam" :int
  (thread :pointer)
  (policy (:pointer :int))
  (param (:pointer sched_param)))
 
(defcfun "pthread_setschedparam" :int
  (thread :pointer)
  (policy :int)
  (param (:pointer sched_param)))
 
 
;;; Lisp interface
(defun get-thread-policy-and-priority (thread)
  (with-foreign-object (policy-ptr :int)
    (with-foreign-object (param-ptr 'sched_param)
      (let* ((result (pthread-getschedparam (make-pointer (sb-thread::thread-os-thread thread))
                                            policy-ptr
                                            param-ptr))
             (policy (mem-ref policy-ptr :int))
             (prio (foreign-slot-value param-ptr 'sched_param 'sched_priority)))
        (declare (fixnum prio policy))
        (values policy prio result)))))
 
(defun set-thread-policy-and-priority (thread policy priority)
  (with-foreign-object (param-ptr 'sched_param)
    (setf (foreign-slot-value param-ptr 'sched_param 'sched_priority) priority)
      (let ((result (pthread-setschedparam (make-pointer (sb-thread::thread-os-thread thread))
                                           policy
                                           param-ptr)))
        (declare (fixnum result))
        result)))
 
 
;;; test
(format t "~A~%" (multiple-value-list (get-thread-policy-and-priority sb-thread:*current-thread*)))
 
(format t "~A~%" (set-thread-policy-and-priority sb-thread:*current-thread* +sched_fifo+ 5))
 
(format t "~A~%" (multiple-value-list (get-thread-policy-and-priority sb-thread:*current-thread*)))

This is by no means a comprehensive solution. In particular POSIX only guarantees a minimum 32 distinct priority values, and the SCHED_OTHER policy doesn’t support non-zero priorities at all. As pointed out in sched_setscheduler(2) (man page for the Linux implementation of sched_setscheduler) the functions sched_get_priority_min and sched_get_priority_max should be used to check the priority range supported by a policy.

You will also note the tiny SBCL-specific bits. To port this to other implementations that use pthreads you need to find out how to get the pointer to the thread’s underlying pthread_t object.

Comments(0)

The POSIX mmap() system call provides a convenient way to access a file’s content.

Basically you just tell the kernel that you want a specific file accessible in memory.

This shifts the responsibility for efficient and organized file access from your application to the kernel. In the meantime you simply work with an in-memory array of the file’s contents.

SBCL provides access to this system call via its SB-POSIX package.

Here’s a small hands-on example using SB-POSIX:MMAP to read a few eight-bit characters from the beginning of my system’s fstab, which starts with this line:

# This file is edited by fstab-sync - see 'man fstab-sync' for details

Let’s start with a basic convenience function in a file.

;;; mmap.lisp
(require :sb-posix)
 
(defun mmap-file (file-stream)
  "Use mmap() to map FILE-STREAM into memory."
  (sb-posix:mmap nil ; where to map (NIL if you don't care)?
                 (file-length file-stream) ; how much?
                 sb-posix:prot-read
                 sb-posix:map-shared
                 (sb-sys:fd-stream-fd file-stream)
                 0)) ; offset into the file

Now we load this file into a fresh SBCL session and work with it:

CL-USER(1): (load "mmap.lisp")
 
; loading system definition from
; /home/sky/projects/lisp/sbcl.git/contrib/sb-grovel/sb-grovel.asd into
; #<PACKAGE "ASDF1">
; registering #<SYSTEM SB-GROVEL {AAC1E19}> as SB-GROVEL
T
CL-USER(2): (mmap-file (open "/etc/fstab"))
 
#.(SB-SYS:INT-SAP #XB7FD3000)
;; This is just a 32-bit pointer to memory; keep it around.
;; Note the readable representation, it's just a wrapped integer.
CL-USER(3): (defparameter p *)
 
P
;; Let's convert this pointer to a so-called "alien".
;; See below for an explanation of what an alien is.
CL-USER(4): (sap-alien p (* char))
 
#<sb -ALIEN-INTERNALS:ALIEN-VALUE :SAP #XB7FD3000 :TYPE (* (SIGNED 8))>
CL-USER(5): (defparameter myalien (sap-alien p (* char)))
 
MYALIEN
;; An alien is just a typed pointer, let's use DESCRIBE to take a look:
CL-USER(6): (describe myalien)
 
#<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #XB7FD3000 :TYPE (* (SIGNED 8))>
  [structure-object]
 
Slots with :INSTANCE allocation:
  SAP   = #.(SB-SYS:INT-SAP #XB7FD3000)
  TYPE  = #<SB-ALIEN-INTERNALS:ALIEN-POINTER-TYPE (* (SIGNED 8))>
;; Now we're ready to access our memory-mapped file. Get the first octet:
CL-USER(7): (deref myalien)
 
35
;; Well, I'm not *that* good at ASCII...
CL-USER(8): (code-char 35)
 
#\#
CL-USER(9): (code-char (deref myalien 1))
 
#\
CL-USER(10): (code-char (deref myalien 2))
 
#\T
CL-USER(11): (code-char (deref myalien 3))
 
#\h
CL-USER(12): (code-char (deref myalien 4))
 
#\i
CL-USER(13): (code-char (deref myalien 5))
 
#\s
;; Looks good.

There are many knobs to fiddle with when you’re working with memory-mapped files. Read mmap(3P) to find out more.

Clozure CL also supports memory-mapped files, see the manual for details.

I hope this post helps newbies to get started with SB-POSIX and SB-ALIEN and helps them to see that it’s not as hard as it might look at first sight.

Sorry for the dodgy escaping of opening braces, Wordpress sucks.

Comments(4)

Weblocks is an advanced web framework written in Common Lisp.

It is designed to make Agile web application development as
effective and simple as possible.

WHY YET ANOTHER WEB FRAMEWORK?

This is not your ordinary run-of-the-mill web framework in PHP, Python or Ruby.

Weblocks uses powerful Lisp features like multiple dispatch, the
metaobject protocol, lexical closures, keyword arguments, and macros
to build abstractions that make web development easy, intuitive, and
free of boilerplate. In addition, control flow is easily expressed
using continuations.

Things that are hard or mundane in other frameworks become easy and
fun in Weblocks. A fine example of this are Weblocks’ AJAX capabilities
which relieves you from writing Javascript in a lot of situations.

AUDIENCE

People who want to get their real-life web programming done
as effectively as possible.

Weblocks is not only targeted at old hands but also at
newcomers to Lisp and Lisp web programming.

It offers a helpful community and code that prevents you
from shooting yourself too easily in the foot.

IS IT USABLE? CAN I SEE SOME DEMOS?

Weblocks is well-tested and has proven its worth in daily usage.
It is used by a community of developers all over the world.

Public applications running Weblocks include

• LAMsight
• Aula Polska
• Thanandar

CHANGES IN 0.8.3

• Greatly enhanced performance in the request handler
• Support for request timeouts
• Automatic bundling/versioning of static files
• Rudimentary profiling
• Fundamentally overhauled nav system
• CLSQL demo fixed
• Updated Simple Blog example
• New HTML error handler
• More flexible debugger configuration with additional restarts
• The usual assortment of fixes and contrib/ additions

MORE INFORMATION

Platforms

Well-tested on SBCL and Clozure CL.
Partially tested on CMUCL, Lispworks, AllegroCL, and OpenMCL.

Official site (with detailed installation guide)

http://weblocks.viridian-project.de/

Demo

http://weblocks.viridian-project.de/weblocks-demo

CONTRIBUTORS

This release has been made possible by Nandan Bagchee, Benjamin Collins,
Stephen Compall, Chris Hallwright, Jan Rychter and yours truly.

Comments(1)