Space Station Lambda

As a precursor to 0.9 which will contain much desired features like AUR system upgrades we decided to release 0.8.3 which contains a bunch of bug fixes:

• Version comparison no longer fails on provider packages (#8)
• Reinstallation works properly again (#7)
• Trying to install or get pkgbuilds for non-existent packages is handled correctly (#5 and #6)
• Question the user in case of malformed pkgbuild dependencies (#12)
• Proxy support works correctly now (#15, reported by nitralime)
• update cache after package removal (proposed by Ralith)
• handle non-Unicode strings more gracefully (#9, reported by zajca)

Syncing up

Use the quickinstall script or sync your Paktahn repo to get it.

Comments(3)

Merry christmas everyone!

It must be christmas of course since Paktahn 0.8.2 is now out as promised! ;D

As it often happens in software development it’s a little later than originally expected, but there’s a lot of good stuff that has made it into this release.

Highlights

• fixed arch=(any) case (reported by magus)
• proper error reporting and restarts when AUR results cannot be fetched (Brit)
• Paktahn now remembers which PKGBUILD files it already presented for review (Brit)
• Paktahn now has proper customizepkg support for AUR packages and will automatically build packages with customizepkg definitions from source (Brit)
• support for just getting a pkgbuild (i.e. yaourt -G) with pak -G pkgnames (Brit)
• makepkg’s PKGDEST variable is detected and used correctly (reported by Stefan Husmann)
• AUR package dependencies are no longer installed explicitly (reported by bram85)
• Basic proxy support (no authentication) (Brit)
• Basic versioning support (Wei Hu, Leslie)

Also, we no longer depend on a custom version of SBCL!

Thanks to Jürgen Hötzel, Wei Hu and of course my colleague Brit Butler for their help with this release.

Syncing up

Use the quickinstall script or sync your Paktahn repo to get it.

Comments(2)

The next version of Paktahn is ready.

Just sync-install your Paktahn package; rc testers will have to remove and reinstall explicitly.

New users should refer to the original blog post for instructions.

Changes

Bug fixes

* fixed x86_64 package building/installation
* fixed case when $EDITOR also had args in it (e.g. “emacsclient -l”)
* throw an error when the user attempts to build an AUR package as root
* fixed –help and -h arguments (reported by Ralith)

New features

* package removal support
* support for provider packages
* support for reinstallation of packages
* ask the user whether he wants to add his arch if it’s not listed in the PKGBUILD.

Comments?

As usual please let me know of any bugs and your wishes.

Comments(2)

[This post is geared towards ArchLinux users. It might confuse you if you're not part of that audience.]

yaourt, the popular Pacman wrapper, doesn’t seem to be maintained anymore and has quite some problems (e.g. UI nits, security issues, huge shell codebase). And people are starting to get tired of it because basic things stop working.

But since yaourt’s interface is so great I have decided to code up an alternative called Paktahn (after a ship type from Wing Commander).

But what can it do?

For now it supports yaourt’s two most helpful (IMHO) operation modes:

pak query

Search AUR and repositories for QUERY in packages names and descriptions and display the results. Ask for one or more packages to be installed.

pak -S package

Install PACKAGE from repositories or AUR.

Paktahn comes with a highly efficient caching system resulting in a speed that will blow your mind if you’re used to Pacman or yaourt.

I’ve been using the current version of Paktahn for weeks now and it has never failed me.

How to get it

Add one of the following sections to your pacman.conf:

# i686
[paktahn]
Server = http://viridian-project.de/~sky/paktahn/repo

# x86_64, package contributed by Gour
[paktahn]
Server = http://viridian-project.de/~sky/paktahn/repo/x86_64

There’s no x86_64 package yet since I cannot build them and no one else has contributed one yet. However you can build it from source: Paktahn on AUR. You need to install a patched version of SBCL before attempting to do so.

Development happens here: http://developer.berlios.de/projects/paktahn/

Discussion thread: http://bbs.archlinux.org/viewtopic.php?id=77674

Contributions, bug reports, comments and praise are all welcome, as are donations since I’m an independent software developer.

Comments(28)

The POSIX threads interface allows user applications to manage scheduling policies and priorities. Certain threads can be made to wait in favor of other threads (priority), and the order of execution for threads of equal priority can be specified (policy).

Below is a humble example interface to access this API.

(asdf:oos 'asdf:load-op 'cffi)
 
(defpackage :prio (:use :cl :cffi))
 
(in-package :prio)
 
;;; constants
;; CFFI-GROVEL offers a more reliable solution for this,
;; but let's keep it simple.
(defconstant +sched_other+ 0)
(defconstant +sched_fifo+ 1)
(defconstant +sched_rr+ 2)
 
 
;;; define the pthread library
(define-foreign-library libpthread
  (:unix "libpthread.so.0"))
 
(use-foreign-library libpthread)
 
 
;;; C structures and functions
(defcstruct sched_param
  (sched_priority :int))
 
(defcfun "pthread_getschedparam" :int
  (thread :pointer)
  (policy (:pointer :int))
  (param (:pointer sched_param)))
 
(defcfun "pthread_setschedparam" :int
  (thread :pointer)
  (policy :int)
  (param (:pointer sched_param)))
 
 
;;; Lisp interface
(defun get-thread-policy-and-priority (thread)
  (with-foreign-object (policy-ptr :int)
    (with-foreign-object (param-ptr 'sched_param)
      (let* ((result (pthread-getschedparam (make-pointer (sb-thread::thread-os-thread thread))
                                            policy-ptr
                                            param-ptr))
             (policy (mem-ref policy-ptr :int))
             (prio (foreign-slot-value param-ptr 'sched_param 'sched_priority)))
        (declare (fixnum prio policy))
        (values policy prio result)))))
 
(defun set-thread-policy-and-priority (thread policy priority)
  (with-foreign-object (param-ptr 'sched_param)
    (setf (foreign-slot-value param-ptr 'sched_param 'sched_priority) priority)
      (let ((result (pthread-setschedparam (make-pointer (sb-thread::thread-os-thread thread))
                                           policy
                                           param-ptr)))
        (declare (fixnum result))
        result)))
 
 
;;; test
(format t "~A~%" (multiple-value-list (get-thread-policy-and-priority sb-thread:*current-thread*)))
 
(format t "~A~%" (set-thread-policy-and-priority sb-thread:*current-thread* +sched_fifo+ 5))
 
(format t "~A~%" (multiple-value-list (get-thread-policy-and-priority sb-thread:*current-thread*)))

This is by no means a comprehensive solution. In particular POSIX only guarantees a minimum 32 distinct priority values, and the SCHED_OTHER policy doesn’t support non-zero priorities at all. As pointed out in sched_setscheduler(2) (man page for the Linux implementation of sched_setscheduler) the functions sched_get_priority_min and sched_get_priority_max should be used to check the priority range supported by a policy.

You will also note the tiny SBCL-specific bits. To port this to other implementations that use pthreads you need to find out how to get the pointer to the thread’s underlying pthread_t object.

Comments(0)

The POSIX mmap() system call provides a convenient way to access a file’s content.

Basically you just tell the kernel that you want a specific file accessible in memory.

This shifts the responsibility for efficient and organized file access from your application to the kernel. In the meantime you simply work with an in-memory array of the file’s contents.

SBCL provides access to this system call via its SB-POSIX package.

Here’s a small hands-on example using SB-POSIX:MMAP to read a few eight-bit characters from the beginning of my system’s fstab, which starts with this line:

# This file is edited by fstab-sync - see 'man fstab-sync' for details

Let’s start with a basic convenience function in a file.

;;; mmap.lisp
(require :sb-posix)
 
(defun mmap-file (file-stream)
  "Use mmap() to map FILE-STREAM into memory."
  (sb-posix:mmap nil ; where to map (NIL if you don't care)?
                 (file-length file-stream) ; how much?
                 sb-posix:prot-read
                 sb-posix:map-shared
                 (sb-sys:fd-stream-fd file-stream)
                 0)) ; offset into the file

Now we load this file into a fresh SBCL session and work with it:

CL-USER(1): (load "mmap.lisp")
 
; loading system definition from
; /home/sky/projects/lisp/sbcl.git/contrib/sb-grovel/sb-grovel.asd into
; #<PACKAGE "ASDF1">
; registering #<SYSTEM SB-GROVEL {AAC1E19}> as SB-GROVEL
T
CL-USER(2): (mmap-file (open "/etc/fstab"))
 
#.(SB-SYS:INT-SAP #XB7FD3000)
;; This is just a 32-bit pointer to memory; keep it around.
;; Note the readable representation, it's just a wrapped integer.
CL-USER(3): (defparameter p *)
 
P
;; Let's convert this pointer to a so-called "alien".
;; See below for an explanation of what an alien is.
CL-USER(4): (sap-alien p (* char))
 
#<sb -ALIEN-INTERNALS:ALIEN-VALUE :SAP #XB7FD3000 :TYPE (* (SIGNED 8))>
CL-USER(5): (defparameter myalien (sap-alien p (* char)))
 
MYALIEN
;; An alien is just a typed pointer, let's use DESCRIBE to take a look:
CL-USER(6): (describe myalien)
 
#<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #XB7FD3000 :TYPE (* (SIGNED 8))>
  [structure-object]
 
Slots with :INSTANCE allocation:
  SAP   = #.(SB-SYS:INT-SAP #XB7FD3000)
  TYPE  = #<SB-ALIEN-INTERNALS:ALIEN-POINTER-TYPE (* (SIGNED 8))>
;; Now we're ready to access our memory-mapped file. Get the first octet:
CL-USER(7): (deref myalien)
 
35
;; Well, I'm not *that* good at ASCII...
CL-USER(8): (code-char 35)
 
#\#
CL-USER(9): (code-char (deref myalien 1))
 
#\
CL-USER(10): (code-char (deref myalien 2))
 
#\T
CL-USER(11): (code-char (deref myalien 3))
 
#\h
CL-USER(12): (code-char (deref myalien 4))
 
#\i
CL-USER(13): (code-char (deref myalien 5))
 
#\s
;; Looks good.

There are many knobs to fiddle with when you’re working with memory-mapped files. Read mmap(3P) to find out more.

Clozure CL also supports memory-mapped files, see the manual for details.

I hope this post helps newbies to get started with SB-POSIX and SB-ALIEN and helps them to see that it’s not as hard as it might look at first sight.

Sorry for the dodgy escaping of opening braces, Wordpress sucks.

Comments(4)

Weblocks is an advanced web framework written in Common Lisp.

It is designed to make Agile web application development as
effective and simple as possible.

WHY YET ANOTHER WEB FRAMEWORK?

This is not your ordinary run-of-the-mill web framework in PHP, Python or Ruby.

Weblocks uses powerful Lisp features like multiple dispatch, the
metaobject protocol, lexical closures, keyword arguments, and macros
to build abstractions that make web development easy, intuitive, and
free of boilerplate. In addition, control flow is easily expressed
using continuations.

Things that are hard or mundane in other frameworks become easy and
fun in Weblocks. A fine example of this are Weblocks’ AJAX capabilities
which relieves you from writing Javascript in a lot of situations.

AUDIENCE

People who want to get their real-life web programming done
as effectively as possible.

Weblocks is not only targeted at old hands but also at
newcomers to Lisp and Lisp web programming.

It offers a helpful community and code that prevents you
from shooting yourself too easily in the foot.

IS IT USABLE? CAN I SEE SOME DEMOS?

Weblocks is well-tested and has proven its worth in daily usage.
It is used by a community of developers all over the world.

Public applications running Weblocks include

• LAMsight
• Aula Polska
• Thanandar

CHANGES IN 0.8.3

• Greatly enhanced performance in the request handler
• Support for request timeouts
• Automatic bundling/versioning of static files
• Rudimentary profiling
• Fundamentally overhauled nav system
• CLSQL demo fixed
• Updated Simple Blog example
• New HTML error handler
• More flexible debugger configuration with additional restarts
• The usual assortment of fixes and contrib/ additions

MORE INFORMATION

Platforms

Well-tested on SBCL and Clozure CL.
Partially tested on CMUCL, Lispworks, AllegroCL, and OpenMCL.

Official site (with detailed installation guide)

http://weblocks.viridian-project.de/

Demo

http://weblocks.viridian-project.de/weblocks-demo

CONTRIBUTORS

This release has been made possible by Nandan Bagchee, Benjamin Collins,
Stephen Compall, Chris Hallwright, Jan Rychter and yours truly.

Comments(1)

The CLHS says the following about DEFPACKAGE:

If the new definition is at variance with the current state of that package, the consequences are undefined; an implementation might choose to modify the existing package to reflect the new definition.

SBCL opts to modify the package but throws a warning, most often in the form of “PACKAGE also exports the following symbols: …”.

Luckily the warning is a specific one, i.e. SB-INT:PACKAGE-AT-VARIANCE.

So to get rid of the warning the following macro helps:

(defmacro without-package-variance-warnings (&body body)
  `(eval-when (:compile-toplevel :load-toplevel :execute)
     (handler-bind (#+sbcl(sb-int:package-at-variance #'muffle-warning))
       ,@body)))

Apply sparsely.

We need the EVAL-WHEN to ensure that the non-toplevel definition of the package is available at compile time.

There’s a minor catch here. One might be tempted to put the EVAL-WHEN clause inside the HANDLER-BIND. But this leads to the package not being available at compile time (therefore rendering the EVAL-WHEN ineffective) because “the compile-time side effects described in Section 3.2 (Compilation) only take place when eval-when appears as a top level form.” (CLHS on EVAL-WHEN).

Does anyone know how to muffle these redefinition warnings in other implementations?

Comments(0)

Usage

(use-package :sb-thread)

Creating a thread

(make-thread (lambda () ...) :name "optional name")

Listing threads

(list-all-threads)

Debugging threads

CL-USER(15): (make-thread (lambda () (break)))
 
debugger invoked on a SIMPLE-CONDITION in thread #< thread RUNNING {AA9E831}>:
  break
 
#< thread RUNNING {AA9E831}>
CL-USER(16): (release-foreground)
 
Resuming thread #< thread RUNNING {A93CD49}>
 
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
 
restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE        ] Return from BREAK.
  1: [TERMINATE-THREAD] Terminate this thread (#< thread RUNNING {A93CD49}>)
 
(BREAK "break")3
0] (release-foreground)
Resuming thread #< thread "initial thread" RUNNING {A6DD551}>
CL-USER(17):

Cleaning up

CL-USER(25): (make-thread (lambda () (unwind-protect (break) (format t "cleanup~%"))))
 
#<thread RUNNING {AB89DB9}>
CL-USER(26):
debugger invoked on a SIMPLE-CONDITION in thread #< thread RUNNING {AB89DB9}>:
  break
(release-foreground)
Resuming thread #< thread RUNNING {AB89DB9}>
 
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
 
restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE        ] Return from BREAK.
  1: [TERMINATE-THREAD] Terminate this thread (#< thread RUNNING {AB89DB9}>)
 
(BREAK "break")
0]
1
Resuming thread #< thread "initial thread" RUNNING {A6DD551}>
1
CL-USER(27): cleanup
</thread>

Miscellaneous notes

Hash tables

Thread model

Comments(1)

The Emacs-based Slime is an excellent IDE for Common Lisp up to the point of inducing people to switch from Vim to Emacs.

Personally I feel that there’s a need for either a full-featured vi written in Common Lisp (no, not GUI-only and written in Python, sorry) or a vi mode for Emacs that is consistent and also full-featured.

Until then you have several possiblities, most of which have been devised pretty recently:

• Nekthuth uses the Swank/Slime model with a library on the Lisp side and a Python scripted Vim plugin on the editor side. It offers a bunch of good things, but I haven’t tried it, yet.
• Limp seems to be the current star among Vim/Lisp bridges, with an active community. I’m going to try this soon.
• Like Slime for Vim is a solution that relies almost solely on GNU Screen. No Hyperspec lookup or function completion without additional work, though.
• Use some additional hints for a comfortable setup.

These approaches have varying implementation support. You’re always fine with SBCL, but Nekthuth, for example, doesn’t support other implementations.

I’m currently using the plain rlwrap approach but I might take advantage of some other approach soon.

Comments(2)