A JSON parser and generator in Common-Lisp

  1. Q&A
  2. User Guide
    1. The default encoder / decoder
    2. The CLOS decoder
    3. Customizing the decoder
    4. Customizing the encoder
    5. Camel case translation
    6. Error handling
    7. Utilities
  3. Mailing Lists
  4. License


What does CL-JSON do?

CL-JSON provides an encoder of Lisp objects to JSON format and a corresponding decoder of JSON data to Lisp objects. Both the encoder and the decoder are highly customizable; at the same time, the default settings ensure a very simple mode of operation, similar to that provided by YASON or ST-JSON.

What is JSON?

JSON is a language independent text format for data-interchange. JSON is especially convenient in web applications, since it is a subset of the literal object notation of ECMAScript. It can also be an alternative to XML. JSON has good open-source support in many languages.

Why not use XML instead?

Some find JSON lighter and more simple, see this comparison.

Why not use s-expressions instead?

How do I download CL-JSON?

A Darcs repository is available.

darcs get

The latest release can be downloaded here.

You can also install it by asdf-install.

History has shown that the darcs version is probably better than the latest release.

User Guide

The default encoder / decoder

The basic functionality is provided by the two functions encode-json and decode-json (all Lisp names in this Manual are available in the package JSON). Under the default semantics they implement the following mapping between Lisp and JSON values:

(capitalized names refer to productions in RFC 4627)
integer Number, with no Frac or Exp parts
float Number, with Frac or Exp parts
the value t true
the value nil null
any other symbol
with identifier name translation
string String
(except Object member keys)
(except association lists)
any other sequence
association list Object
(in implementations with MOP)
interned in *json-symbols-package*
with identifier name translation
String as an Object member key

[generic function] encode-json object &optional stream

Write a JSON representation of object to stream and return nil.

[function] decode-json &optional stream

Read a JSON Value from stream and return the corresponding Lisp value.

If stream is omitted it defaults to *json-output* for the encoder, *json-input* for the decoder. The initial values of these variables are synonym streams to *standard-output* and *standard-input* respectively.

[variable] *json-output*

The default output stream for encoding operations.

[variable] *json-input*

The default input stream for decoding operations.

  '#( ((foo . (1 2 3)) (bar . t) (baz . #\!))
      "quux" 4/17 4.25))

Some parameters which influence the encoding and decoding of symbols are:

[variable] *json-symbols-package*

The package where JSON Object keys etc. are interned. Default keyword, nil = use current *package*.

    (s "{\"foo\": [1, 2, 3], \"bar\": true, \"baz\": \"!\"}")
  (json:decode-json s))
((:FOO 1 2 3) (:BAR . T) (:BAZ . "!"))
    (s "{\"foo\": [1, 2, 3], \"bar\": true, \"baz\": \"!\"}")
  (let ((json:*json-symbols-package* 'json-test))
    (json:decode-json s)))    
((JSON-TEST::FOO 1 2 3) (JSON-TEST::BAR . T) (JSON-TEST::BAZ . "!"))
    (s "{\"foo\": [1, 2, 3], \"bar\": true, \"baz\": \"!\"}")
  (let ((json:*json-symbols-package* nil))
    (json:decode-json s)))
((FOO 1 2 3) (BAR . T) (BAZ . "!"))

[variable] *json-identifier-name-to-lisp*

Designator for a function which maps string (a JSON Object key) to string (name of a Lisp symbol).

[variable] *lisp-identifier-name-to-json*

Designator for a function which maps string (name of a Lisp symbol) to string (e. g. JSON Object key).

The last two variables are initially bound to #'camel-case-to-lisp and #'lisp-to-camel-case, respectively. For a description of these functions, see Camel case translation below.

Note also that the format of Lisp floats decoded from JSON input is controlled by the system variable *read-default-float-format* in the same manner as for the Lisp reader. (But cf. Error handling.)

CL-JSON also provides the following shortcut functions:

[function] encode-json-to-string object

Return the JSON representation of object as a string.

[function] encode-json-alist alist &optional stream

Write the JSON representation (Object) of alist to stream (or to *json-output*). Return nil.

[function] encode-json-alist-to-string alist

Return the JSON representation (Object) of alist as a string.

[function] encode-json-plist plist &optional stream

Write the JSON representation (Object) of plist to stream (or to *json-output*). Return nil.

[function] encode-json-plist-to-string plist

Return the JSON representation (Object) of plist as a string.

[function] decode-json-strict &optional stream

Same as decode-json, but allow only Objects or Arrays on the top level, no junk afterwards.

[function] decode-json-from-string json-string

Read a JSON Value from json-string and return the corresponding Lisp value.

[function] decode-json-from-source source &optional decoder

Decode a JSON Value from source using the value of decoder (default 'decode-json) as decoder function. If the source is a string, the input is from this string; if it is a pathname, the input is from the file that it names; otherwise, a stream is expected as source.

The CLOS decoder

It is possible to switch CL-JSON into a mode where Arrays are decoded to Lisp vectors rather than lists, and Objects are decoded to CLOS objects in the manner described below. Switching between this mode (“CLOS semantics”) and the default “list semantics” is done using the following functions:

[function] set-decoder-simple-clos-semantics

[function] set-decoder-simple-list-semantics

Either semantics can be made to hold temporarily in a dynamic environment while body is being executed:

[macro] with-decoder-simple-clos-semantics &body body

[macro] with-decoder-simple-list-semantics &body body

As a general rule, under the CLOS semantics a JSON Object datum D is decoded to an instance F of an anonymous class K. Such a class does not have a name which could be used as a method specializer, however, the user can always assume that K is a subclass of fluid-object and an instance of the metaclass fluid-class; that it has at least the slots named by the keys in D (which are decoded to symbols in the usual manner); and that in F these slots are bound to Lisp images of the corresponding JSON Values, and any other slot is unbound:

  (let ((json:*json-symbols-package* nil))
    (let ((x (json:decode-json-from-string
               "{\"foo\": [1, 2, 3], \"bar\": true,
                 \"baz\": \"!\"}")))
      (with-slots (foo bar baz) x
        (values x foo bar baz)))))
#(1 2 3)

Beyond this, CL-JSON provides a means for the remote party to specify the class (or the superclasses) of the decoded CLOS object. This is done by supplying the JSON Object with appropriate metadata, called that Object's prototype. The prototype is the Member of the Object whose key matches *prototype-name*—more precisely, whose key decodes to a symbol S such that (string= (symbol-name S) (symbol-name json:*prototype-name*)).

[variable] *prototype-name*

Must have a symbol for value, the default is json:prototype. If the value is nil then no member of an Object is to be treated as the prototype (not even one whose key otherwise matches nil).

Note: The default prototype name, prototype has been chosen so that it would not, in all likelihood, interfere with programs' naming schemes in a JavaScript remote party. As this name has special meaning within JavaScript, it is unlikely that it would be used for a user-level member. Users of CL-JSON are, of course, free to assign another name that better suits their own needs.

The Value of the prototype must be either a String, or an Object whose (undecoded) keys include "lispPackage", "lispClass", and "lispSuperclasses". All Members with other keys are ignored. The Value for any of "lispPackage", "lispClass", and "lispSuperclasses" may be null which is equivalent to omitting that Member. A prototype S which is a String (rather than an Object) is equivalent to a prototype that has only the Member "lispClass":S. An Object altogether lacking a prototype is equivalent to it having a prototype with all the three meaningful members omitted. With the foregoing exceptions, the Values for "lispPackage" and "lispClass" should be Strings, and the Value for "lispSuperclasses" an Array of Strings.

The Value for "lispPackage" is decoded as symbol and passed to find-package to obtain a package, P. The Value for "lispClass", Members of the Value for "lispSuperclasses", and the keys of the encompassing Object are then decoded as symbols in the package P. (If "lispPackage" is null or missing, they are interned according to the generic rule, i. e. in *json-symbols-package*.)

If the prototype has a "lispClass" member, its decoded value must name an existing class K. The encompassing Object, D, is then decoded to an instance of K. The Values of all Members of D whose keys, when decoded, yield names of slots in K are assigned to those slots. All other Members are discarded. Also, if K has a slot whose name is the same as *prototype-name* that slot shall be bound to an instance of prototype which describes the prototype in the JSON Object.

(in-package cl-user)
(defclass sample () (foo bar))
  (let ((x (json:decode-json-from-string
            "{\"foo\": [1, 2, 3], \"bar\": true, \"baz\": \"!\",
              \"prototype\": {\"lispPackage\": \"clUser\",
                              \"lispClass\": \"sample\"}}")))
      (with-slots (foo bar) x
        (values x (class-of x) foo bar (slot-exists-p x 'baz)))))
#(1 2 3)

By way of exception, if the decoded value for "lispClass" is common-lisp:hash-table, common-lisp:cons, or common-lisp:list then the encompassing Object shall be decoded, respectively, to a hash table, an association list, or a property list with the keys and values decoded from the keys and values of the encompassing Object. (The prototype member itself shall be discarded.)

(let ((json:*json-symbols-package* 'cl-user))
   (let ((x (json:decode-json-from-string
             "{\"foo\": [1, 2, 3], \"bar\": true, \"baz\": \"!\",
               \"prototype\": \"hashTable\"}")))
     (loop for v being each hash-value of x
       collect v))))
("!" #(1 2 3) T) ; or in another order
    "{\"foo\": [1, 2, 3], \"bar\": true, \"baz\": \"!\",
      \"prototype\": {\"lispPackage\": \"jsonTest\",
                      \"lispClass\": \"list\"}}"))

If the prototype has no "lispClass" but has "lispSuperclasses", the Value of the latter is decoded as a list of symbols which must name existing classes L1, ... Ln, n ≥ 0. The encompassing Object, D, is then decoded to a fluid object F whose class K has all the classes L1, ... Ln as its direct superclasses. Unless any of the classes L1, ... Ln is a subclass of fluid-object, K shall be additionally a direct subclass of fluid-object. Slots corresponding to all Members of D (except the prototype itself) shall be bound in F.

(defclass sample-2 ()
  ((frob :initform 'xyzzy)))
  (let ((json:*json-symbols-package* nil))
    (let ((x (json:decode-json-from-string
               "{\"foo\": [1, 2, 3], \"bar\": true, \"baz\": \"!\",
                    {\"lispSuperclasses\": [\"sample2\"]}}")))
      (with-slots (foo frob) x
        (values x foo frob)))))
#(1 2 3)

If the prototype has both "lispClass" and "lispSuperclasses", the latter member is disregarded.

[standard class] prototype (precedence list: prototype, standard-object, t)

[generic function] make-object-prototype object &optional slot-names

Return a prototype describing the object's class or superclasses, and the package into which the names of the class / superclasses and of the object's slots are to be interned.

Properties of fluid objects

Fluid objects are quite like standard objects in most respects. However, any fluid object has fluid-class as its metaclass, and missing slots in such fluid instances are not distinguished from unbound slots. Namely, if F is a fluid instance of class K, and R is a slot name such that (slot-exists-p F R) gives nil then:

This makes fluid objects somewhat like objects in prototype-based languages, in that the user may add new slots to an object without concerning himself with the object's class membership. Yet fluid objects are little more that “poor man's JavaScript” as there are no real prototype mechanisms. Moreover, Lisp implementations usually employ optimizations geared toward more traditional CLOS techniques which can actually cause less than ideal performance in programs with fluid objects. Thus, fluid objects should generally be considered an experimental feature and used with proper care.

[standard class] fluid-object (precedence list: fluid-object, standard-object, t)

[standard class] fluid-class (precedence list: fluid-class, standard-class, class, standard-object, t)

The CLOS decoder maintains a separate fluid class for each combination of superclasses in prototypes sent in by remote parties. I. e., if the prototypes of two incoming JSON Objects D1 and D2 have "lispSuperclasses" members whose values decode to non-equal lists of symbols then the fluid objects F1 and F2 decoded from D1 and D2 shall belong to different fluid classes. This behaviour is implemented by the generic function make-object.

[generic function] make-object bindings class &optional superclasses

If class is not nil, create an instance of that class. Otherwise, create a fluid object whose class has the given superclasses (null list by default). In either case, populate the resulting object using bindings (an alist of slot names and values).

To maintain the mapping between lists of superclass names and fluid classes, the decoder maintains a class registry. Thus, using fluid objects makes the CLOS decoder essentially thread-unsafe. (If every incoming JSON Object is guaranteed to have a prototype with a "lispClass" member then there are no fluid objects and thread safety is ensured.) If the user wishes to employ fluid objects in a threaded environment it is advisable to wrap the body of entry-point functions in with-local-class-registry.

[variable] *class-registry*

A list of anonymous fluid classes, one member for every distinct combination of direct superclasses.

[function] clear-class-registry

Reset the *class-registry* to nil.

[macro] with-local-class-registry (&key inherit) &body body

Run body in a dynamic environment where *class-registry* is a temporary local list. If :inherit is non-null, the local registry shall initially have the same content as the exterior *class-registry*, otherwise it shall be nil.

Customizing the decoder

More arbitrary changes to decoder semantics can be made via the customization API. It is an event-driven API not unlike SAX: certain variables can be set (or dynamically bound) to handler functions which are invoked, in sequence, as the parser encounters various syntactic elements of JSON on the input stream. JSON well-formedness is checked by the underlying parser, while the handlers are free to dispose of the (known clean) data in any way the user may find fit.

Handlers fall into three overall categories: constructors, which can be further subdivided into atomic constructors and aggregate constructors; aggregate initializers; and member handlers. An atomic constructor is invoked when the parser encounters an atomic datum (Number or Boolean), is passed the token which constitutes the datum, and is supposed to return the Lisp value that corresponds to the datum. An aggregate initializer is invoked when the parser encounters the beginning of an aggregate datum (String, Array or Object). Similarly, an aggregate constructor is invoked at the end of a well-formed aggregate datum, and is supposed to return the Lisp value that corresponds to the whole aggregate. A member handler is invoked when the parser reads past a member of an Array, or a key or value of an Object; it is passed the value which the constructor for that member datum had freshly returned. Handlers for Characters of Strings are somewhat specific in that they are passed characters rather than decoded values but otherwise they fall neatly with member handlers. The value of the last constructor to return is also the return value of decode-json.

As a trivial example, if every handler has the value of (constantly t), the decoder semantics is that of a simple validator: it returns t if JSON input is well-formed, and signals error otherwise (see also under Error handling).

Macros which set or bind handlers (and other customization parameters) take their arguments in the form of property lists where each customizaion parameter is indicated by its distinctive keyword. E. g., :integer and :boolean are the respective indicator keywords for the variables *integer-handler* and *boolean-handler*. Accordingly, the macro form (json:bind-custom-vars (:boolean X :integer Y) Z) expands to (let ((json:*boolean-handler* X) (json:*integer-handler* Y)) Z). Descriptions of customization parameters below always specify the corresponding keyword, and descriptions of macro APIs use the parameter name customizations whereever a property list is expected.

[macro] with-shadowed-custom-vars &body body

Execute body in a dynamic environment where the names of all existing customization parameters are re-bound to fresh locations. The initial values of the parameters are taken from the exterior environment at runtime.

[macro] bind-custom-vars (&rest customizations) &body body

Execute body in a dynamic environment where customization parameters are bound as specified by customizations.

[macro] set-custom-vars &rest customizations

Globally set customization parameters according to customizations. If used within a dynamic environment established by with-shadowed-custom-vars or bind-custom-vars, the extent of the assignment is limited by that form.

The following parameters identify the constructor handlers for atomic data:

[custom variable] *boolean-handler* (indicator: :boolean)

Designator for a function of 1 string argument (boolean token).

[custom variable] *integer-handler* (indicator: :integer)

Designator for a function of 1 string argument (integer token).

[custom variable] *real-handler* (indicator: :real)

Designator for a function of 1 string argument (real token).

For Strings, three handlers (an initializer, a member handler, and a constructor) are used:

[custom variable] *beginning-of-string-handler* (indicator: :beginning-of-string)

Designator for a function of no arguments (called at encountering an opening quote for a String).

[custom variable] *string-char-handler* (indicator: :string-char)

Designator for a function of 1 character argument (String char).

[custom variable] *end-of-string-handler* (indicator: :end-of-string)

Designator for a function of no arguments (called at encountering a closing quote for a String).

Note that escape sequences in Strings are resolved by the underlying decoder, and a *string-char-handler* normally receives nothing but characters. However, limitations of the Lisp implementation running CL-JSON can prevent it from resolving a perfectly well-formed escape sequence (e. g., in CMUCL which does not support Unicode). Situations of this kind are discussed below in the section Error handling.

For Arrays, three handlers are used as well:

[custom variable] *beginning-of-array-handler* (indicator: :beginning-of-array)

Designator for a function of no arguments (called at encountering an opening bracket for an Array).

[custom variable] *array-member-handler* (indicator: :array-member)

Designator for a function of 1 arbitrary argument (decoded member of Array).

[custom variable] *end-of-array-handler* (indicator: :end-of-array)

Designator for a function of no arguments (called at encountering a closing bracket for an Array).

For Objects, it is the same thing except that there are two member handlers, one for keys and another one for values, invoked alternately:

[custom variable] *beginning-of-object-handler* (indicator: :beginning-of-object)

Designator for a function of no arguments (called at encountering an opening brace for an Object).

[custom variable] *object-key-handler* (indicator: :object-key)

Designator for a function of 1 string argument (decoded member key of Object).

[custom variable] *object-value-handler* (indicator: :object-value)

Designator for a function of 1 arbitrary argument (decoded member value of Object).

[custom variable] *end-of-object-handler* (indicator: :end-of-object)

Designator for a function of no arguments (called at encountering a closing brace for an Object).

If the user wished to map JSON structure to Lisp structure, with only these 13 handlers he would have to maintain cumbersome stacking infrastructure, pushing at the beginning and popping at the end of every aggregate datum. CL-JSON has a means to circumvent this hindrance: the user is given an option to indicate any number of dynamic variables V1, ... Vn as having aggregate scope. This has the effect that the ensemble of handlers invoked for any particular aggregate datum on the input are called in a dynamic environment where the names V1, ... Vn are bound to fresh locations. Hence, an aggregate-scope variable can be initialized by an initializer handler, updated by member handlers, and finally used to calculate the return value of the constructor. If several aggregate data are nested, the handlers for interior data work on their own copy of the variable without fear of clobbering the value of the same for exterior data. Handler code can be thus written in terms of individual levels rather than complex nested structures.

[custom variable] *aggregate-scope-variables* (indicator: :aggregate-scope)

A list of symbols naming dynamic variables which should be re-bound in the scope of every JSON aggregate value (Object, Array or String).

Aggregate-scope variables on inner levels initially inherit their values from the immediately outer level, so it is possible to pass information down from level to level. In the following example, the variable count is reset to 0 at the beginning of every aggregate, while level is incremented from outer to inner levels.

(defparameter level -1
  "Level of nesting in JSON structure.")
(defparameter count -1
  "Count of members in JSON structure.")
(defun format-with-indent (control &rest args)
  (format t "~&~0,1,V,V@A~?" level #\Tab "" control args))
(defun report-atom (token)
  (format-with-indent "~Ctoken: ~S~%" #\Tab token)
(defun report-push ()
  (incf level)
  (setq count 0)
  (format-with-indent "beginning of aggregate~%"))
(defun report-member (member)
  (format-with-indent "~:R member: ~A~%"
                      (incf count) member))
(defun report-pop ()
  (format-with-indent "end of aggregate~%")
  (list 'aggregate count))
    (:integer              #'report-atom
     :real                 #'report-atom
     :boolean              #'report-atom
     :beginning-of-array   #'report-push
     :array-member         #'report-member
     :end-of-array         #'report-pop
     :beginning-of-string  #'report-push
     :string-char          #'report-atom
     :end-of-string        #'report-pop
     :beginning-of-object  #'report-push
     :object-key           (constantly t)
     :object-value         #'report-member
     :end-of-object        #'report-pop
     :aggregate-scope      (list 'level 'count))
    "{\"foo\": [10, 20, 50], \"bar\": true}"))
beginning of aggregate
	beginning of aggregate
		token: #\f
		token: #\o
		token: #\o
	end of aggregate
	beginning of aggregate
		token: "10"
	first member: ATOM
		token: "20"
	second member: ATOM
		token: "50"
	third member: ATOM
	end of aggregate
first member: (AGGREGATE 3)
	beginning of aggregate
		token: #\b
		token: #\a
		token: #\r
	end of aggregate
	token: "true"
second member: ATOM
end of aggregate

Further, it is possible to re-bind variables while parsing some kinds of aggregate data but not the others:

[custom variable] *array-scope-variables* (indicator: :array-scope)

A list of symbols naming dynamic variables which should be re-bound in the scope of every JSON Array.

[custom variable] *object-scope-variables* (indicator: :object-scope)

A list of symbols naming dynamic variables which should be re-bound in the scope of every JSON Object.

[custom variable] *string-scope-variables* (indicator: :string-scope)

A list of symbols naming dynamic variables which should be re-bound in the scope of every JSON String.

When using one of the standard semantics, the following parameter allows to determine the type of sequence to which Arrays are decoded. The default value is list for the list semantics, and vector for the CLOS semantics.

[custom variable] *json-array-type* (indicator: :array-type)

The Lisp sequence type to which JSON Arrays are to be coerced.

Finally, the user can change the decoder function for member data of the current aggregate. The name *internal-decoder* is included in the default list of *aggregate-scope-variables* under both predefined semantics, and so this variable can be assigned locally without affecting the decoding of the outer levels of data. If it is set outside of any handler, the value is used to decode all members of top-level aggregates.

[custom variable] *internal-decoder* (indicator: :internal-decoder)

Designator for a function of 1 stream argument called (instead of decode-json) to decode a member of an Array or of an Object.

The following macros can be used to construct decoder functions acceptable as values of *internal-decoder*:

[macro] custom-decoder &rest customizations

Construct a decoder function whose semantics is determined by the given customizations.

[macro] current-decoder &rest keys

Capture the current (execution-time) decoder semantics and construct a decoder function which restores the captured semantics in the extent of its call. The keys must be valid indicators of custom variables. If some keys are supplied then the values of only the corresponding variables are captured; otherwise the values of all 19 custom variables are captured.

The following example demonstrates how a simplified version of the macro json-bind can be written with the help of *internal-decoder*.

(defmacro simple-json-bind ((&rest vars) stream &body body)
  (let ((cur-dec (gensym))
         `(lambda (json-string)
            (let ((lisp-string
                   (funcall json:*json-identifier-name-to-lisp*
              ;; On recognizing a variable name, the key handler sets
              ;; the value handler to be a function which,
              ;; in its turn, assigns the decoded value to the
              ;; corresponding variable.  If no variable name matches
              ;; the key, the value handler is set to skip the value.
                      ,@(loop for var in vars
                           `((string= lisp-string ,(symbol-name var))
                             (lambda (value)
                               (setq ,var value))))
                      (t (constantly t))))))))
    `(let ((,cur-dec (json:current-decoder))
           (:internal-decoder ,cur-dec
            :beginning-of-object (constantly t)
            :object-key ,key-handler
            :end-of-object (constantly t))
         (json:decode-json ,stream))
  '(simple-json-bind (foo bar) s
     (format t "Foo is ~A.~%Bar is ~A.~%" foo bar))))
      FOO BAR)
           (LET ((LISP-STRING
                   (COND ((STRING= LISP-STRING "FOO")
                          (LAMBDA (VALUE) (SETQ FOO VALUE)))
                         ((STRING= LISP-STRING "BAR")
                          (LAMBDA (VALUE) (SETQ BAR VALUE)))
                         (T (CONSTANTLY T))))))
  (FORMAT T "Foo is ~A.~%Bar is ~A.~%" FOO BAR))
      (s "{\"foo\": [1, 2, 3], \"baz\": {\"foo\": \"!\"},
           \"bar\": true}")
    (simple-json-bind (foo bar) s
      (format t "Foo is ~A.~%Bar is ~A.~%" foo bar))))
Foo is (1 2 3).
Bar is T.

Customizing the encoder

Encoding being much more user-driven than decoding, the customization API for the encoder boils down to just a few macros and functions, geared to generating aggregate data. They implement a sequential producer which emits JSON data upon user request, rather than in the course of traversing Lisp data structures. As with encode-json, the optional stream argument in these macros and functions defaults to *json-output*.

[macro] with-array (&optional stream) &body body

Open a JSON Array, run body, then close the Array. Inside the body, as-array-member or encode-array-member should be called to encode Members of the Array.

[macro] as-array-member (&optional stream) &body body

Body should be a program which encodes exactly one JSON datum to stream. As-array-member ensures that the datum is properly formatted as a Member of an Array, i. e. separated by comma from any preceding or following Member.

[function] encode-array-member object &optional stream

Encode object as the next Member of the innermost JSON Array opened with with-array in the dynamic context. Object is encoded using the encode-json generic function, so it must be of a type for which an encode-json method is defined.

[function] stream-array-member-encoder stream &optional encoder

Return a function which takes an argument and encodes it to stream as a Member of an Array. The encoding function is taken from the value of encoder (default is #'encode-json).

[macro] with-object (&optional stream) &body body

Open a JSON Object, run body, then close the Object. Inside the body, as-object-member or encode-object-member should be called to encode Members of the Object.

[macro] as-object-member (key &optional stream) &body body

Body should be a program which writes exactly one JSON datum to stream. As-object-member ensures that the datum is properly formatted as a Member of an Object, i. e. preceded by the (encoded) key and colon, and separated by comma from any preceding or following Member.

[function] encode-object-member key value &optional stream

Encode key and value as a Member pair of the innermost JSON Object opened with with-object in the dynamic context. Key and value are encoded using the encode-json generic function, so they both must be of a type for which an encode-json method is defined. If key does not encode to a String, its JSON representation (as a string) is encoded over again.

[function] stream-object-member-encoder stream &optional encoder

Return a function which takes two arguments and encodes them to stream as a Member of an Object (String : Value pair).

(defpackage foo (:use) (:export #:foo #:bar #:baz))
(defvar foo:foo '(1 2 3))
(defvar foo:bar "!")
(defun foo:bar ()
(json:with-array ()
  (json:as-array-member ()
    (json:with-object ()
      (do-external-symbols (sym (find-package "FOO"))
        (json:as-object-member (sym)
          (json:with-object ()
            (if (boundp sym)
                (let ((value (symbol-value sym)))
                  (json:encode-object-member 'val value)))
            (if (fboundp sym)
                (json:encode-object-member 'fun t))))))))

Camel case translation

The aim of the functions described in this section is to provide a reasonable mapping between identifier naming conventions—from JavaScript to Lisp and back again. Names are regarded as consisting of pieces that, in Lisp, are always uppercase and joined with hyphens, and in JavaScript (and other C-like languages), either joined with underscores or capitalized and joined solid. Plus signs are used in Lisp to mark constants, which corresponds to the all-caps convention in C-like languages.

[function] camel-case-to-lisp string

[function] lisp-to-camel-case string

The functions are not strictly inverse. While (lisp-to-camel-case (camel-case-to-lisp S)) is string= to S for every S which is a valid JavaScript identifier name, (camel-case-to-lisp (lisp-to-camel-case R)) is not guaranteed to preserve R even if R is a valid Lisp identifier name. The use of camel-case-to-lisp and lisp-to-camel-case is best explained by some examples:

(json:lisp-to-camel-case "HELLO-KEY")
(json:camel-case-to-lisp "HiStartsWithUpperCase")
(json:camel-case-to-lisp "joined_by_underscore")
(json:camel-case-to-lisp "JSONAllCapitals")
(json:lisp-to-camel-case "+TWO-WORDS+")
(json:camel-case-to-lisp "camelCase_Mixed_4_PARTS")

Error handling

The standard encoder does not cover the complete set of Lisp data types: structures, functions, complex numbers, and cons pairs which are not lists and not members of a valid alist are only some of the unencodable data types. Attempting to encode any such data results in an error.

[condition] unencodable-value-error (precedence list: unencodable-value-error, type-error, error, serious-condition, condition, t)

Signalled when a datum is passed to encode-json (or another encoder function) which actually cannot be encoded. As with any type-error, the accessors type-error-datum and type-error-expected-type can be used on a condition of this class to get the offending datum and the expected type (however, the latter is always t).

Whenever an unencodable-value-error is signalled, CL-JSON provides a restart called substitute-printed-representation which, if invoked, prints a representation of the offending datum to a string (with *print-escape* bound to nil) and encodes that string instead of the datum.

(defstruct frob xyzzy)
      (lambda (err)
        (declare (ignore err))
        (invoke-restart 'json:substitute-printed-representation))))
    '(#S(frob) ((foo . #C(1 1)) (bar . (baz . quux))))))
["#S(FROB :XYZZY NIL)",{"foo":"#C(1 1)","bar":"(BAZ . QUUX)"}]

The encoder also signals an error if any macro or function for encoding Members of aggregates (such as as-object-member) is called outside of dynamic environment established by with-array or with-object, or in inappropriate environment.

More numerous are situations in which an error is caused by input to the decoder. The most straightforward case is when data sent in by the remote party is not well-formed JSON. This results in a json-syntax-error being signalled. (Some syntax errors, e. g. missing right brackets, result in end-of-file instead.)

[condition] json-syntax-error (precedence list: json-syntax-error, simple-error, simple-condition, stream-error, error, serious-condition, condition, t)

However, even well-formed JSON is not guaranteed to decode. Two common kinds of error situations result from implementation restrictions on character codes and on floating-point reals. If the decoder encounters in a JSON string an escape sequence \uXXXX where XXXX is the hexadecimal presentation of a number N which either exceeds the implementation's char-code-limit or for which code-char returns nil, then the following condition is signalled:

[condition] no-char-for-code (precedence list: no-char-for-code, error, serious-condition, condition, t)

In this situation a handler for no-char-for-code may invoke the following restarts:

substitute-char char
Continue the execution as if the given char, rather than the escaped char, had been read.
Continue the execution using the code itself in place of the character. The code is subsequently passed to the *string-char-handler*, so using this restart only has sense if the handler groks character codes (the standard handler doesn't).

Likewise, the underlying Lisp system can fail to handle floats which are not representable in the current *read-default-float-format* (e. g. 2.065e444 is non-representable even in the long-float format in most contemporary Lisps). If such a datum is encountered in JSON input, an arithmetic-error is signalled. CL-JSON provides three restarts to correct such cases:

bignumber-string &optional prefix
Instead of number, use a string which is the concatenation of prefix and the number token that caused the error. The default value for prefix is "BIGNUMBER:".
Parse the number token into parts, read them as integers, and combine the integers using only rational operations. Use the resulting rational number in place of float. (E. g., 2.065e444 would be converted to the value of the expression (* (+ 2 (* 65 (expt 10 -3))) (expt 10 444)).)
placeholder value
Use value in place of the float. E. g., one might wish to use :infinity if the error in question is a float-overflow.

The CLOS decoder can run into errors due to invalid data in a prototype:

These situations, as well as json-syntax-error are considered fatal, and so no restarts are provided by CL-JSON.



[macro] json-bind (&rest vars) json-source &body body

The stream, string, or file json-source should give off a JSON object hierarchy for which vars are structured keys. The Values corresponding to these keys are decoded using the current decoder, vars are bound to the decoded values, and body is executed in the resulting lexical environment.

By “object hierarchy”, we understand a tree of nested JSON Objects (without intervening Arrays). Structured keys are symbols whose names consist of parts separated by periods and serve as paths into this tree. In an Object D, one-part name S (without any periods) refers to the Value E whose corresponding key R is such that (string= (funcall *json-identifier-to-lisp* R) S). If E is also an Object, the path S.T refers to the same sub-datum within D which T refers to within E. If any part of the structured name fails to match a member key at its level of object nesting, or if any non-final part of the structured name refers to a non-Object, then that variable shall be bound to nil in the scope of json-bind.

(let ((dentition
            {\"incisors\":2, \"canines\":0,
             \"premolars\":3, \"molars\":3},
            {\"incisors\":3, \"canines\":1,
             \"premolars\":3, \"molars\":3}},
            {\"incisors\":4, \"canines\":1,
             \"premolars\":2, \"molars\":4},
            {\"incisors\":3, \"canines\":1,
             \"premolars\":3, \"molars\":4}},
          {\"upper\":6, \"lower\":8}}"))
    (json:json-bind (*notoryctes.lower.incisors
      (flet ((count-teeth (jaw)
              (* (loop for (k . v) in jaw summing v) 2)))
        (format t "Armadillos have ~D teeth.~%"
                (count-teeth *dasypus-hybridus))
        (format t "Marsupial moles have ~D teeth in the lower jaw.~%"
                (count-teeth *notoryctes.lower)))
      (let ((delta
              (- *notoryctes.lower.incisors
        (format t "Marsupial moles have ~[less~;as many~;more~] ~
                   lower incisors ~:[than~;as~] dugongs.~%"
                (1+ (signum delta)) (zerop delta))))))
Armadillos have 28 teeth.
Marsupial moles have 22 teeth in the lower jaw.
Marsupial moles have as many lower incisors as dugongs.



Mailing Lists


Copyright © 2006-2008 Henrik Hjelte.
Copyright © 2008 Hans Hübner.

This work is licensed under the terms of the MIT / X Consortium license, reproduced below. Mr. Hübner's copyright applies to the code of the streaming API borrowed from his program YASON and transferred under this license with his permission.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.