Major highlights:

• Support for AUR updates: pak -Su --aur
• Support for .tar.xz packages
• Support for custom Paktahn directory

More details can be found on Brit’s blog.

To upgrade or install just use the quick installer.

• Version comparison no longer fails on provider packages (#8)
• Reinstallation works properly again (#7)
• Trying to install or get pkgbuilds for non-existent packages is handled correctly (#5 and #6)
• Question the user in case of malformed pkgbuild dependencies (#12)
• Proxy support works correctly now (#15, reported by nitralime)
• update cache after package removal (proposed by Ralith)
• handle non-Unicode strings more gracefully (#9, reported by zajca)

Syncing up

Use the quickinstall script or sync your Paktahn repo to get it.

It must be christmas of course since Paktahn 0.8.2 is now out as promised! ;D

As it often happens in software development it’s a little later than originally expected, but there’s a lot of good stuff that has made it into this release.

Highlights

• fixed arch=(any) case (reported by magus)
• proper error reporting and restarts when AUR results cannot be fetched (Brit)
• Paktahn now remembers which PKGBUILD files it already presented for review (Brit)
• Paktahn now has proper customizepkg support for AUR packages and will automatically build packages with customizepkg definitions from source (Brit)
• support for just getting a pkgbuild (i.e. yaourt -G) with pak -G pkgnames (Brit)
• makepkg’s PKGDEST variable is detected and used correctly (reported by Stefan Husmann)
• AUR package dependencies are no longer installed explicitly (reported by bram85)
• Basic proxy support (no authentication) (Brit)
• Basic versioning support (Wei Hu, Leslie)

Also, we no longer depend on a custom version of SBCL!

Thanks to Jürgen Hötzel, Wei Hu and of course my colleague Brit Butler for their help with this release.

Syncing up

Use the quickinstall script or sync your Paktahn repo to get it.

It will add the Paktahn binary repository to /etc/pacman.conf and run Pacman to install Paktahn.

wget -qO- http://viridian-project.de/\~sky/paktahn/quickinstall.sh | sudo sh

You just need to paste this line into a terminal. Review the script here, it’s very simple:

http://viridian-project.de/~sky/paktahn/quickinstall.sh

Just sync-install your Paktahn package; rc testers will have to remove and reinstall explicitly.

New users should refer to the original blog post for instructions.

Changes

Bug fixes

* fixed x86_64 package building/installation
* fixed case when $EDITOR also had args in it (e.g. “emacsclient -l”)
* throw an error when the user attempts to build an AUR package as root
* fixed –help and -h arguments (reported by Ralith)

New features

* package removal support
* support for provider packages
* support for reinstallation of packages
* ask the user whether he wants to add his arch if it’s not listed in the PKGBUILD.

Comments?

As usual please let me know of any bugs and your wishes.

Paktahn 0.8.1 is ready but has only been tested by me, so here’s a prerelease for you to run and help find bugs:

http://viridian-project.de/~sky/paktahn/rc/paktahn-0.8.1-1-i686.pkg.tar.gz

Changes:

* package removal support
* support for provider packages
* support reinstallation of packages
* throw an error when the user attempts to build an AUR package as root

You’ve already been very supportive with your feedback for 0.8! Thanks!

yaourt, the popular Pacman wrapper, doesn’t seem to be maintained anymore and has quite some problems (e.g. UI nits, security issues, huge shell codebase). And people are starting to get tired of it because basic things stop working.

But since yaourt’s interface is so great I have decided to code up an alternative called Paktahn (after a ship type from Wing Commander).

But what can it do?

For now it supports yaourt’s two most helpful (IMHO) operation modes:

pak query

Search AUR and repositories for QUERY in packages names and descriptions and display the results. Ask for one or more packages to be installed.

pak -S package

Install PACKAGE from repositories or AUR.

Paktahn comes with a highly efficient caching system resulting in a speed that will blow your mind if you’re used to Pacman or yaourt.

I’ve been using the current version of Paktahn for weeks now and it has never failed me.

How to get it

Add one of the following sections to your pacman.conf:

# i686
[paktahn]
Server = http://viridian-project.de/~sky/paktahn/repo

# x86_64, package contributed by Gour
[paktahn]
Server = http://viridian-project.de/~sky/paktahn/repo/x86_64

There’s no x86_64 package yet since I cannot build them and no one else has contributed one yet. However you can build it from source: Paktahn on AUR. You need to install a patched version of SBCL before attempting to do so.

Development happens here: http://developer.berlios.de/projects/paktahn/

Discussion thread: http://bbs.archlinux.org/viewtopic.php?id=77674

Contributions, bug reports, comments and praise are all welcome, as are donations since I’m an independent software developer.

The Twitter web interface allows you to take a look at older tweets in bunches of about a twenty each. And every time you want to fetch a new batch you have to click on the large “more” button. Which in turn often brings up the message features in this blog post’s title.

I’m annoyed as hell by this of course, but it also gives me the opportunity for a little analysis that will end in the conclusion: Twitter will fail (unless they take some spoons of Tech Clue and change radically, that is).

What’s behind the little error message? You can’t really say for sure what happened, but since these errors occur pretty randomly it’s probably just that their servers are very busy right now.

Plus Twitter is notoriously known for having problems with scalability. For roughly the last two years, in fact. If you ask me they must be pretty clueless about scalability issues. To my knowledge their last action to solve this was to switch to another programming language (Scala). This approach seems to be motivated more by the name of the language than the actual benefit derived from switching to it. I don’t know if they have made the switch yet or whether they changed their strategy, but chances are their scalability problems won’t be solved at all with this.

The only thing that could cure their scalability worries would be a proper protocol for distributed twittering. There are suitable and proven distributed protocols like XMPP. You know, that’s the thing the Jabber instant messaging network is built on. And Google Wave too. This is quite a clue on the ability of Wave to supersede Twitter. Instead the latter ones are wasting their time with fruitless efforts. Come on!

This is made worse by the sheer simplicity of their application. I mean, in one of my projects I’m the sole developer of a complex game application. It’s understandable that I can’t be all things for all people, and scalability is just one of many problems. But what the hell are they doing all day at Twitter? They have 29 employees right now (source: TechCrunch)! Are they all busy keeping the list of top tweets fresh, or making Excel diagrams to please their VCs?

What’s your opinion on Twitter, scalability and the future of micro-blogging?

Below is a humble example interface to access this API.

(asdf:oos 'asdf:load-op 'cffi)
 
(defpackage :prio (:use :cl :cffi))
 
(in-package :prio)
 
;;; constants
;; CFFI-GROVEL offers a more reliable solution for this,
;; but let's keep it simple.
(defconstant +sched_other+ 0)
(defconstant +sched_fifo+ 1)
(defconstant +sched_rr+ 2)
 
 
;;; define the pthread library
(define-foreign-library libpthread
  (:unix "libpthread.so.0"))
 
(use-foreign-library libpthread)
 
 
;;; C structures and functions
(defcstruct sched_param
  (sched_priority :int))
 
(defcfun "pthread_getschedparam" :int
  (thread :pointer)
  (policy (:pointer :int))
  (param (:pointer sched_param)))
 
(defcfun "pthread_setschedparam" :int
  (thread :pointer)
  (policy :int)
  (param (:pointer sched_param)))
 
 
;;; Lisp interface
(defun get-thread-policy-and-priority (thread)
  (with-foreign-object (policy-ptr :int)
    (with-foreign-object (param-ptr 'sched_param)
      (let* ((result (pthread-getschedparam (make-pointer (sb-thread::thread-os-thread thread))
                                            policy-ptr
                                            param-ptr))
             (policy (mem-ref policy-ptr :int))
             (prio (foreign-slot-value param-ptr 'sched_param 'sched_priority)))
        (declare (fixnum prio policy))
        (values policy prio result)))))
 
(defun set-thread-policy-and-priority (thread policy priority)
  (with-foreign-object (param-ptr 'sched_param)
    (setf (foreign-slot-value param-ptr 'sched_param 'sched_priority) priority)
      (let ((result (pthread-setschedparam (make-pointer (sb-thread::thread-os-thread thread))
                                           policy
                                           param-ptr)))
        (declare (fixnum result))
        result)))
 
 
;;; test
(format t "~A~%" (multiple-value-list (get-thread-policy-and-priority sb-thread:*current-thread*)))
 
(format t "~A~%" (set-thread-policy-and-priority sb-thread:*current-thread* +sched_fifo+ 5))
 
(format t "~A~%" (multiple-value-list (get-thread-policy-and-priority sb-thread:*current-thread*)))

This is by no means a comprehensive solution. In particular POSIX only guarantees a minimum 32 distinct priority values, and the SCHED_OTHER policy doesn’t support non-zero priorities at all. As pointed out in sched_setscheduler(2) (man page for the Linux implementation of sched_setscheduler) the functions sched_get_priority_min and sched_get_priority_max should be used to check the priority range supported by a policy.

You will also note the tiny SBCL-specific bits. To port this to other implementations that use pthreads you need to find out how to get the pointer to the thread’s underlying pthread_t object.

Basically you just tell the kernel that you want a specific file accessible in memory.

This shifts the responsibility for efficient and organized file access from your application to the kernel. In the meantime you simply work with an in-memory array of the file’s contents.

SBCL provides access to this system call via its SB-POSIX package.

Here’s a small hands-on example using SB-POSIX:MMAP to read a few eight-bit characters from the beginning of my system’s fstab, which starts with this line:

# This file is edited by fstab-sync - see 'man fstab-sync' for details

Let’s start with a basic convenience function in a file.

;;; mmap.lisp
(require :sb-posix)
 
(defun mmap-file (file-stream)
  "Use mmap() to map FILE-STREAM into memory."
  (sb-posix:mmap nil ; where to map (NIL if you don't care)?
                 (file-length file-stream) ; how much?
                 sb-posix:prot-read
                 sb-posix:map-shared
                 (sb-sys:fd-stream-fd file-stream)
                 0)) ; offset into the file

Now we load this file into a fresh SBCL session and work with it:

CL-USER(1): (load "mmap.lisp")
 
; loading system definition from
; /home/sky/projects/lisp/sbcl.git/contrib/sb-grovel/sb-grovel.asd into
; #<PACKAGE "ASDF1">
; registering #<SYSTEM SB-GROVEL {AAC1E19}> as SB-GROVEL
T
CL-USER(2): (mmap-file (open "/etc/fstab"))
 
#.(SB-SYS:INT-SAP #XB7FD3000)
;; This is just a 32-bit pointer to memory; keep it around.
;; Note the readable representation, it's just a wrapped integer.
CL-USER(3): (defparameter p *)
 
P
;; Let's convert this pointer to a so-called "alien".
;; See below for an explanation of what an alien is.
CL-USER(4): (sap-alien p (* char))
 
#<sb -ALIEN-INTERNALS:ALIEN-VALUE :SAP #XB7FD3000 :TYPE (* (SIGNED 8))>
CL-USER(5): (defparameter myalien (sap-alien p (* char)))
 
MYALIEN
;; An alien is just a typed pointer, let's use DESCRIBE to take a look:
CL-USER(6): (describe myalien)
 
#<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #XB7FD3000 :TYPE (* (SIGNED 8))>
  [structure-object]
 
Slots with :INSTANCE allocation:
  SAP   = #.(SB-SYS:INT-SAP #XB7FD3000)
  TYPE  = #<SB-ALIEN-INTERNALS:ALIEN-POINTER-TYPE (* (SIGNED 8))>
;; Now we're ready to access our memory-mapped file. Get the first octet:
CL-USER(7): (deref myalien)
 
35
;; Well, I'm not *that* good at ASCII...
CL-USER(8): (code-char 35)
 
#\#
CL-USER(9): (code-char (deref myalien 1))
 
#\
CL-USER(10): (code-char (deref myalien 2))
 
#\T
CL-USER(11): (code-char (deref myalien 3))
 
#\h
CL-USER(12): (code-char (deref myalien 4))
 
#\i
CL-USER(13): (code-char (deref myalien 5))
 
#\s
;; Looks good.

There are many knobs to fiddle with when you’re working with memory-mapped files. Read mmap(3P) to find out more.

Clozure CL also supports memory-mapped files, see the manual for details.

I hope this post helps newbies to get started with SB-POSIX and SB-ALIEN and helps them to see that it’s not as hard as it might look at first sight.

Sorry for the dodgy escaping of opening braces, WordPress sucks.