Partial MP3 decoder in Haskell

Here’s a partially conforming MP3 decoder in Haskell.

It looks pretty clean even to me, not being a Haskell programmer.

Who’s up for it in Common Lisp? :)

Threads in SBCL

Working with threads in SBCL isn't obvious in some respects. I'd like to use this post to collect some hints for people starting out with multi-threaded programming in SBCL. Prerequisite: SBCL compiled with threads (still not the default). Check with (not (null (member :sb-thread *features*))).

Usage

All thread stuff lies in the SB-THREAD package, so you need to import it when you don't want to do prefixing:
(use-package :sb-thread)

Creating a thread

(make-thread (lambda () ...) :name "optional name")
Spawns a new thread executing the lambda and return it. The thread will silently terminate when the lambda returns. You can wait for its completion using JOIN-THREAD.

Listing threads

(list-all-threads)
'nuff said.

Debugging threads

Use RELEASE-FOREGROUND to switch between multiple threads waiting for input:
CL-USER(15): (make-thread (lambda () (break)))
 
debugger invoked on a SIMPLE-CONDITION in thread #< thread RUNNING {AA9E831}>:
  break
 
#< thread RUNNING {AA9E831}>
CL-USER(16): (release-foreground)
 
Resuming thread #< thread RUNNING {A93CD49}>
 
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
 
restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE        ] Return from BREAK.
  1: [TERMINATE-THREAD] Terminate this thread (#< thread RUNNING {A93CD49}>)
 
(BREAK "break")3
0] (release-foreground)
Resuming thread #< thread "initial thread" RUNNING {A6DD551}>
CL-USER(17):

Cleaning up

Perhaps this is obvious to someone familiar with the stack semantics of threading:
CL-USER(25): (make-thread (lambda () (unwind-protect (break) (format t "cleanup~%"))))
 
#<thread RUNNING {AB89DB9}>
CL-USER(26):
debugger invoked on a SIMPLE-CONDITION in thread #< thread RUNNING {AB89DB9}>:
  break
(release-foreground)
Resuming thread #< thread RUNNING {AB89DB9}>
 
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
 
restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE        ] Return from BREAK.
  1: [TERMINATE-THREAD] Terminate this thread (#< thread RUNNING {AB89DB9}>)
 
(BREAK "break")
0]
1
Resuming thread #< thread "initial thread" RUNNING {A6DD551}>
1
CL-USER(27): cleanup
</thread>

Miscellaneous notes

Hash tables

Somewhere in time SBCL dropped locking for hash tables to give the user the freedom to implement the level of locking they need. So you need to protect your hash tables for concurrent access or else.

Thread model

There are two fundamental models for a thread framework: rely on the native OS facilities for threading or implement some sort of concurrency scheduling yourself. Allegro uses the latter method while SBCL and ECL use native threading.

Using Vim with Common Lisp

The Emacs-based Slime is an excellent IDE for Common Lisp up to the point of inducing people to switch from Vim to Emacs.

Personally I feel that there’s a need for either a full-featured vi written in Common Lisp (no, not GUI-only and written in Python, sorry) or a vi mode for Emacs that is consistent and also full-featured.

Until then you have several possiblities, most of which have been devised pretty recently:

  • Nekthuth uses the Swank/Slime model with a library on the Lisp side and a Python scripted Vim plugin on the editor side. It offers a bunch of good things, but I haven’t tried it, yet.
  • Limp seems to be the current star among Vim/Lisp bridges, with an active community. I’m going to try this soon.
  • Like Slime for Vim is a solution that relies almost solely on GNU Screen. No Hyperspec lookup or function completion without additional work, though.
  • Use some additional hints for a comfortable setup.

These approaches have varying implementation support. You’re always fine with SBCL, but Nekthuth, for example, doesn’t support other implementations.

I’m currently using the plain rlwrap approach but I might take advantage of some other approach soon.

Reflecting on one’s coding self

A surprise incentive to resume blog writing from Aaron Feng, heh.

How old were you when you first started programming?

Ten or eleven.

How did you get started in programming?

Read books about it, peeked at other people’s code, got in front of a computer, started hacking. :)

What was your first language?

BASIC.

What was the first real program you wrote?

A BASIC program calculating the Least Common Multiple of two integers.
On paper.

What languages have you used since you started programming?

BASIC, Turbo Pascal, C, C++, Java, C#, Common Lisp, Scheme, Bourne Shell, Awk, Perl, x86 Assembler, MIPS Assembler, TeX, JavaScript, PHP, SQL.

What was your first professional programming gig?

Depends on what you’d call “professional”.

When I was about 15 years old I wrote a dBase3 to CSV converter in Turbo Pascal for which I received 20 DM (about 10 EUR).

If you knew then what you know now, would you have started programming?

You bet.

If there is one thing you learned along the way that you would tell new developers, what would it be?

Don’t be content with your skills and work. Critically examine your own and others’ results.
Push your limits, push, push, push.

What’s the most fun you’ve ever had … programming?

Me and a friend hacking up a space game prototype with GCC and Allegro at night. I slowly discovered the joys of GNU/Linux and free software at that time.

Writing a boot loader in Assembler.

Letting creative impulses flow into Lisp code with little hindrance.

Who’s next?

Michael Prinzinger
Stephen Compall

I’d also tag Sebastian Heberer but he ain’t got a running blog right now.

Collecting ASDF system dependencies

Here’s a snippet that will recursively collect all dependencies of one or more ASDF systems:

(asdf:oos 'asdf:load-op 'asdf)
(asdf:oos 'asdf:load-op 'metatilities)
 
(defun direct-dependencies (component)
  (cdadr (asdf:component-depends-on 'asdf:load-op
                                    (asdf:find-component nil component))))
 
(defun normalize-system-id (id)
  (intern (symbol-name id) "KEYWORD"))
 
(defun %effective-dependencies (components)
  "Helper function."
  (when components
    (remove-duplicates
      (append components
              (%effective-dependencies
                (mapcar #'normalize-system-id
                        (remove-if #'null
                                   (metatilities:flatten
                                     (mapcar #'direct-dependencies
                                             components))))))
      :test #'eq)))
 
(defun effective-dependencies (components)
  "Find all dependencies needed for the list of COMPONENTS
(which may be an atom, too)."
  (let ((components (metatilities:ensure-list components)))
    (set-difference (%effective-dependencies components) components
                    :key #'normalize-system-id)))
 
;; usage
(effective-dependencies :weblocks)
 
(:LW-COMPAT :CLOSER-MOP :MOPTILITIES :METATILITIES-BASE :TRIVIAL-GRAY-STREAMS
 :ASDF-SYSTEM-CONNECTIONS :METABANG-DYNAMIC-CLASSES :FLEXI-STREAMS :CFFI
 :SB-GROVEL :CL-CONTAINERS :METABANG-BIND :CHUNGA :CL-BASE64 :CL-FAD :CL-PPCRE
 :CL+SSL :MD5 :RFC2388 :SB-BSD-SOCKETS :SB-POSIX :URL-REWRITE :PARENSCRIPT
 :FARE-UTILS :METATILITIES :HUNCHENTOOT :CL-WHO :CL-JSON :PURI :FARE-MATCHER
 :CL-CONT :ITERATE)

Smart dates in CL

Sometimes it’s convenient to present dates in a way that depends on their offset from the current time.

For example, in different resolutions: 23 seconds ago, one minute ago, two days ago.

Another example, making use of human naming conventions: yesterday, Monday (implicitly assuming the closest Monday before the current date).

In Common Lisp, without further babbling:

(load "time.lisp")
;; http://cybertiggyr.com/gene/pdl/time.lisp
;; you could also use, for example, CL-L10N.
 
(defmacro base-bind (unit-var amount (&rest var-and-radix) &body code)
  "Thanks to Alan Crowe for this wonderful macro."
  (if (endp var-and-radix)
    `(let ((,unit-var ,amount)) ,@code)
    (let ((transfer (gensym)))
      `(multiple-value-bind (,transfer ,unit-var)
         (floor ,amount ,(cadar var-and-radix))
         (base-bind ,(caar var-and-radix) ,transfer ,(cdr var-and-radix)
                    ,@code)))))
 
(defun smart-date (then)
  (let ((now (get-universal-time)))
    (base-bind now-sec now ((now-min 60) (now-hour 60) (now-day 24))
      (base-bind then-sec then ((then-min 60) (then-hour 60) (then-day 24))
        (base-bind diff-sec (- now then) ((diff-min 60) (diff-hour 60) (diff-day 24))
          (cond
            ;; add more stuff here (e.g. negative offsets) and modify to suit your needs
            ((> diff-day 6) (CYBERTIGGYR-TIME:format-time nil CYBERTIGGYR-TIME:*FORMAT-TIME-FULL* then))
            ((> diff-day 1) (CYBERTIGGYR-TIME:format-time nil "%A" then))
            ((= diff-day 1) "Yesterday")
            ((> diff-hour 0) (format nil "~Dh~Dm ago" diff-hour diff-min))
            ((> diff-min 0) (format nil "~Dm~Ds ago" diff-min diff-sec))
            (t (format nil "~D seconds ago" diff-sec))))))))
 
; demonstration/test
(loop for offset in (list 36 90 120 130 3599 3600 3601 86400 86500 173000 14290010)
      do (format t "~D: ~A~%" offset (smart-date (- (get-universal-time) offset))))
 
; output:
36: 36 seconds ago
90: 1m30s ago
120: 2m0s ago
130: 2m10s ago
3599: 59m59s ago
3600: 1h0m ago
3601: 1h0m ago
86400: Yesterday
86500: Yesterday
173000: Tuesday
14290010: Sunday, 2008 January 27, 03:06 +1

Again, I’d like to see solutions from other languages.

Batch-renaming files

I needed to number all image files in the current directory:

# current:
16090-04.png       PySolFC_1.png
h3teampysol_20080303103813.jpg  pysol.gif
pysol_420_2.gif 175928_large.jpeg  Pysol.jpg
linux-game-pysol03.png          pysol460_big2.jpg
 
# goal:
pysolfc1.png  pysolfc2.jpeg  pysolfc3.gif
pysolfc4.jpg  pysolfc5.png  pysolfc6.jpg
pysolfc7.png  pysolfc8.gif  pysolfc9.jpg

Surprisingly easy in Common Lisp:

(loop for file in (directory "*")
     for i from 1
     do (rename-file file (format nil "pysolfc~D" i)))

In plain sh programming, I would have had to extract directory component, basename and extension, modify the basename without the extension and put them all together again. And keep track of the index manually, too.

I suppose some shell guru might come up with a neat (and probably quite unreadable) solution, but I’d rather stick to Common Lisp.

I’m curious about other solutions, esp. in Shell (ZSH allowed!), Python, Ruby and less-known languages.

Porting Perl’s qq to Common Lisp

Perl has the useful qq operator which lets you specify an arbitrary delimiter for the string following it:

# here with exclamation mark:
qq!we "often" use "quotes" "here".!

In Common Lisp, this would be useful as well, especially in docstrings and when generating foreign language code (think JavaScript without Parenscript, for example).

Let over Lambda shows us the useful sharp-doublequote reader macro that lets #" and "# act as delimiters.

This already helps a lot and looks very good, but sometimes you have a lot of double quotes and the terminating combination "# inside one string.

Take a look at this piece of JQuery code in Lisp (CL-WHO html generation):

(:a :onclick (format nil "$(\"#content\").load(\"~A.clhp\"); return true;" id))

Sharp-doublequote won’t work here because of "#. We can’t use single quotes here either because Hunchentoot will delimited the onclick part with them. We could probably add a space between " and #, but it would be a kludge and might not work in other cases anyway.

The bottom line is that using a fixed character or character combination won’t work for all cases (except when the delimiter is really long like MIME boundary strings, but this is obviously impractical).

So letting the user choose the delimiter on a case-by-case basis is a smart decision (as long as it is not overused and clutters the code with all sorts of delimiters).

The following code provides this functionality:

(defun |#q-reader| (stream sub-char numarg)
  (declare (ignore sub-char numarg))
  (let ((terminator (read-char stream)))
    (loop for ch = (read-char stream)
          until (eql ch terminator)
          collect ch into chars
          finally (return (coerce chars 'string)))))
 
(set-dispatch-macro-character
    #\# #\q #'|#q-reader|)

Quick test:

% clisp -repl qq.lisp
[1]> "foo"
"foo"
[2]> #q|foo|
"foo"
[3]> #q|foo bar baz|
"foo bar baz"
[4]> #q!foo bar baz!
"foo bar baz"
[5]> #q!foo bar "baz!
"foo bar \"baz"
[6]> #q!Hello! world! ; oops
"Hello"
[7]>
*** - SYSTEM::READ-EVAL-PRINT: variable WORLD! has no value

errormarker.vim: a Vim plugin for C/C++ programming

ErrorMarker.vim will gather all errors reported by the GCC C and C++ compilers after a :make command.

Shadowing a CL function definition

Shadowing functions is useful for example when testing.
Suppose we want to build a tiny test suite around the following function:

 
(defparameter *appointment* ...)
 
(defun overdue-p ()
  (>= (get-universal-time) *appointment*))

Testing OVERDUE-P obviously requires us to test two branches: one where the appointment is overdue and one where it is not.

Let’s say that you can’t change *APPOINTMENT* in your testing context for whatever reasons (the reason here being that this example is ultra-contrived for simplicity).
Assuming a fixed value for *APPOINTMENT* we need to change the return value of CL:GET-UNIVERSAL-TIME. This can be achieved by shadowing this function for the duration of our test.

Unfortunately, shadowing a function in Common Lisp isn’t obvious.

You can’t use FLET or LABELS because they have lexical scope.
You can’t use DEFUN either because it affects the global function namespace and doesn’t let you save or restore the old definition.

The only way I know of is using the function (SETF FDEFINITION):

(let ((orig (fdefinition 'get-universal-time)))
  (setf (fdefinition 'get-universal-time) (lambda () *my-testing-time*))
  (prog1
    (overdue-p) ; you'd run some test checks against the result here
    (setf (fdefinition 'bar) orig)))

Wrapping this in a macro is left as an exercise to the reader.

This doesn’t work for special operators, and neither for FLET or LABELS. See the CLHS entry for accessor FDEFINITION.

« Previous PageNext Page »