← Back to Posts

Common Lisp Bindings

Oct 07, 2013 - by Mike McMahon

In preparing for the upcoming release of the FoundationDB SQL layer, I needed to conduct some performance tests and interoperability experiments. But let's back up for a second, in case you don't know FoundationDB is a key-value NoSQL database that combines scalability, fault-tolerance, and high performance with multi-key ACID transactions. The SQL Layer is a full ANSI SQL engine that runs on top of the key-value store.

Now, I wanted access to the store that allowed for rapid prototyping. FoundationDB has fully supported bindings for C, Java, Python, Ruby, JavaScript (Node.js) and C# (.Net). But I wasn't in the mood for quirky syntax or complicated scoping semantics. So I wrote unsupported bindings for Common Lisp.

This is not an enormous amount of work, because the C API does all the network communication and is accessed using CFFI. This makes these Lisp bindings reasonably portable. They have been tested on SBCL and Clozure CL.

(defpackage :fdb-user
  (:use #:common-lisp #:foundationdb))
(in-package :fdb-user)
(api-version 100)
(defvar *db* (database-open))
(with-transaction (tr *db*)
  (setf (transaction-get tr "hello") "world"))
(with-transaction (tr *db*)
  (future-value (transaction-get tr "hello")))

As you can see, futures are exposed in asynchronous results. Additionally, there is a simple form of completion callback that should be sufficient to integrate with higher level synchronization.

(with-transaction (tr *db*)
  (let ((futures (map 'list #'(lambda (key) (transaction-get tr key))
                      '("a" "b" "c" "d")))
        (semaphore (sb-thread:make-semaphore)))
    (flet ((wake-up (future)
             (declare (ignore future))
             (sb-thread:signal-semaphore semaphore)))
      (dolist (future futures)
        (setf (future-callback future) #'wake-up)))
    (loop while futures do
      (sb-thread:wait-on-semaphore semaphore)
      ;; Handle as many as are now ready.
      (setq futures (delete-if #'(lambda (future)
                                   (when (future-ready-p future)
                                     (print (babel:octets-to-string (future-value future)))

            For iterating over query results, there is support for both DO and MAP style.

(with-transaction (tr *db*)
  (do-range-query ((key value) tr (range-starts-with "h"))
    (format t "~&~S~T~S~%" key value)))
(with-transaction (tr *db*)
  (map-range-query 'list #'(lambda (key value) 
                             (concatenate 'string (babel:octets-to-string key)
                                           " -- " (babel:octets-to-string value)))
                   tr "a" "z" :reverse-p t))

LOOP keywords should not be hard to add, though they would, of course, be less portable in so far as LOOP implementations differ.

      There is compatible support for the higher-level tuple and directory layers.

      The bindings are meant to be reasonably idiomatic and that has been preferred over strict consistency with any other language bindings. But I am perfectly willing to hear that the style is a quarter century out of date and to entertain suggestions for updates.

      These bindings can be found on GitHub. If there is interest, we can get them added to Quicklisp. We are also happy to accept pull requests for supporting other ANSI Common Lisp systems, just let us know!

Download Key-Value Store

Topics: Key-Value Store

Subscribe to Email Updates