Lisp II

If the cell is divided by a line, as is shown at right, then this indicates two different examples.

Lisp arrays are created with the function make-array. The simplest form of this function is:

(make-array length)

This form makes a one-dimensional fixed-length array length elements long. The elements are each initialized to nil.

An array of this form is called a simple-vector. You don't just have to use make-array to build a simple-vector. Just as you can make a list of the symbols a b c by typing '(a b c), you can make a simple vector of the symbols a b c by typing #(a b c)

(make-array dimension-list)

This form makes an N-dimensional fixed-length array of the dimensions given by elements in the list. The elements are each initialized to nil.

You can specify the initial value of the elements with the keyword :initial-element.

The general function for extracting the element of any array is aref. It takes the form:

(aref array index1 index2 ...)

Simple vectors have a special version, svref, which is slightly faster than aref (in fact, aref just calls svref for simple vectors):

(svref simple-vector index)

Lisp arrays are zero-indexed. This is just like saying (in C++/Java): array[index1][index2]...

Multidimensional arrays can also be specified with #nA(...), where n is the number of dimensions. See the example at right.

Variable-length vectors are created with the keywords :adjustable and :fill-pointer in the following fashion:

(make-array length :fill-pointer t :adjustable t)

You can have a zero-length vector. It's very common to start a variable-length array at length 0.

You can tack new stuff onto the end of a variable-length vector with the command vector-push-extend. You can "pop" elements off the end of the variable-length vector with vector-pop.

To use these functions, the vector must be variable-length. You cannot push and pop to a simple vector.

Multidimensional arrays can also have their sizes adjusted. We'll just leave it at that -- look it up if you're interested.

Although string elements can be accessed via aref, strings have their own special function which does the same thing: char, which takes the form:

(char string index)

In most systems, the two functions are about the same speed.

To set the value of an element in an array (I bet you were wondering about that!) you say

(setf (aref array indices... ) val)

You can do the same trick with svref and char.

You can also use setf to modify lists. However, this is dangerous if you don't know what you're doing. For now, don't do it. Stick with modifying arrays and strings.

(incf expression 4)

...will see to it that expression evaluates to 4 more than it used to (by adding 4 to it). If you just say:

(incf expression)

...this by default sees to it that expression evaluates to 1 more than it used to.

The macro decf does the opposite.

(push val expression)

...this is roughly the same as saying

(setf expression (cons val expression))

You can also "see to it" that a list has an element removed from the front of it with pop:

(pop expression)

(rotatef expresion1 expression2 ... expressionN)

...this is roughly the same as saying

(setf tempvar expression1)
(setf expression1 expression2)
...
(setf expressionN-1 expressionN)
(setf expressionN tempvar)

A simple use of this is simply (rotatef expression1 expression2) which sees to it that the values of expression1 and expression2 are swapped.

The special form function will return a pointer to a function. It takes the form (function function-symbol). Notice that just like quote, function doesn't evaluate its argument -- instead it just looks up the function by that name and returns a pointer to it.

Also like quote, function is so common that there is a shorthand for it: a pound sign followed by a quote at the beginning of the function name:

#'print

...is the same as...

(function print)

Keep in mind that you can only get pointers to functions, not macros or special forms.

Remember that the first item in an evaluated list must be a symbol which is not evaluated. If a variable could be put as the first item, it would have to be evaluated first (to extract the function pointer).

Thus, Common Lisp can associate a function with a symbol (by using defun) and it can also associate a value with the same symbol as a variable (by using setf). A Lisp which can associate two or more different kinds of things at the same time with a symbol is called a Lisp 2. Common Lisp is a Lisp 2. Emacs Lisp is also a Lisp 2.

Scheme, another popular Lisp dialect, evaluates the first item in the list as a variable, looking up its function-pointer value. Scheme associates only one thing with a symbol: the item stored in its variable. Thus Scheme is a Lisp 1.

Lisp 1's are simpler and more intuitive than Lisp 2's. But it is more difficult to do certain kinds of powerful things with them, like macros. We'll get to that later on.

There are a great many functions and macros which use function pointers. One basic one is funcall. This function takes the form

(funcall function-pointer arg1 arg2 ... )

funcall is a function which evaluates function-pointer, which returns a pointer to a function, then it evaluates each of the arguments, then passes the argument values into the function. funcall returns the value of the function.

(apply function-pointer list-arg )

apply takes a function pointer, plus one more argument which must evaluate to a list. It then takes each element in this list and passes them as arguments to the function pointed to by function-pointer. apply then returns the value the that the function returned.

It so happens that apply can do one additional trick. Alternatively, apply can look like this: (apply function-pointer arg1 arg2 ... list-arg )

The last argument must evaluate to a list. Here, apply builds a list before passing it to the function. This list is built by taking each of the arg1, arg2, arguments and concatenating their values to the front of the list returned by list-arg. For example, in (apply #'+ 1 2 3 '(4 5 6)), the concatenation results in the list '(1 2 3 4 5 6). Thus

(apply #'+ 1 2 3 '(4 5 6)) is the same thing as

(apply #'+ '(1 2 3 4 5 6)) which is the same thing as

(apply #'+ 1 2 3 4 5 6 '()) which of course is the same thing as

(apply #'+ 1 2 3 4 5 6 nil)

One very common use of pointers to functions is mapping. Mapping applies a function repeatedly over one or more lists, resulting in a new list. The most common mapping function is mapcar, which in a basic form looks like this:

(mapcar function-pointer list)

Since we're providing just one list, function-pointer must be a pointer to a function which can take just one argument, for example, sqrt.

In this form, mapcar repeatedly applies the function to each element in the list. The return values are then put into a list and returned.

mapcar more generally looks like this:

(mapcar function-pointer list1 list2 ...)

If function-pointer points to a function which takes N arguments, then we must provide N lists.

mapcar takes the first element out of each list and passes them as arguments to the function. mapcar then takes the second element out of each list and passes them as arguments to the function. And so on. mapcar then returns a list of the return values of the function.

If any list is shorter than the others, mapcar operates only up to the shortest list and then stops.

Lisp provides a number of other useful mapping functions: map, mapc, mapcan, mapcon ...

A related feature is reduction: composing a function in on itself. The basic reduce function looks similar to mapcar:

(reduce function-pointer list)

function-pointer must point to a function which takes exactly two arguments. If the elements in list are a b c d, and the function func is stored in the function pointer, this is the same thing as doing:

(func (func (func a b) c) d)

You can also change the order of operations with the :from-end t keyword argument, resulting in the ordering:

(func a (func b (func c d)))

reduce has other gizmos available. Check 'em out.

A lambda expression is one of the more powerful concepts in Lisp. A lambda expression is an anonymous function, that is one that doesn't have a name -- just a pointer to it.

Lambda expressions are created using the form:

(function (lambda (args...) body...))

Note how similar this is to defun:

(defun function-name (args...) body...)

A lambda expression builds a function just like defun would, except that there's no name associated with it. Instead, the lambda expression returns a pointer to the function.

Remember that function has a shorthand of #' so the lambda expression is usually written like this:

#'(lambda (args...) body...)

To make things even more confusing, Common Lisp has provided for you an actual macro called lamda, which does exactly the same thing. Thus if you really want to (but it's not good style) you can write it as just:

(lambda (args...) body...)

Lambda expressions are useful when you need to pass in a quick, short, temporary function. But there is another very powerful use of lambda expressions: making closures.

A closure is a function bundled together with its own lexical scope. Usually you can think of this as a closure being a function plus its own personal, private global variables.

When a function is built from a lambda expression, it is usually created in the context of some outer local variables. After the function is built, these variables are "trapped" with the lambda expression if anything in the lambda expression referred to them. Since the lambda expression is hanging on to these variables, they're not garbage collected when the local scope is exited. Instead they become private variables that only the function can see.

We can use this concept to make function-building functions. Consider:

(defun build-a-function (x)
       #'(lambda (y) (+ x y)))

...examine this function carefully. build-a-function takes a value x and then returns a function which adds that amount x to things!

Closures are also common when we need to make a quick custom function based on information the user provided. Consider:

(defun add-to-list (val list-of-numbers)
  (mapcar #'(lambda (num) (+ val num))
          list-of-numbers))

...examine this function carefully as well. add-to-list takes a number val and a list of numbers. It then maps a custom function on the list of numbers. This custom function adds val to each one. The new list is then returned.

Notice that the lambda expression is converted into a function even though it refers to val inside the lambda expression.

Closures are examples of powerful things which C++ simply cannot do. Java gets there part-way. Java can do lambda expressions in the form of "anonymous classes". But it too cannot do real closures, though there are nasty hacks to work around the issue.

Closures also occur with defun. Imagine if defun were called inside a let statement:

(let ((seed 1234))
  (defun rand ()
    (setf seed (mod (* seed 16807) 2147483647))))

Here we defined a local variable called seed. Inside this local environment, we defined a function called rand which uses seed. When we leave the let, what happens to seed? Normally it would get garbage collected. But it can't here -- because rand is holding on to it. seed becomes a private global variable of the function rand. No one else can see it but rand.

You can use this for other interesting purposes. Imagine that you want to make a private bank account:

(let ((account 0))
  (defun deposit ($$$)
    (setf account (+ account $$$)))
  (defun withdraw ($$$)
    (setf account (- account $$$)))
  (defun amount ()
    account))

The functions deposit, withdraw, and amount share a common private variable called account that no one else can see.

This isn't much different from a Java or C++ object with a private instance variable and three methods. Where did you think object-oriented programming came from? You got it.

In fact, Lisp can be easily modified to do rather OOP built on top of closures. It comes with an OOP system, CLOS, as part of the language (though I think CLOS is too mammoth, so I usually make my own little OOP language in Lisp instead).

A function which works with any kind of sequence is a sequence function (duh). We've seen some examples of sequence functions before: length, reverse, subseq.

Another common sequence function is elt, of the form:

(elt sequence index)

elt returns element #index in the sequence. You can use elt inside setf to set the element (again, don't change elements in lists unless you know what you're doing. Strings and vectors are fine).

elt is an example of a general function: it works with a variety of data types, but as a result is slower than custom-made functions for each data type. For example, if you know your sequence is a string, aref is probably faster. If you know your sequence is a simple-vector, svref is much faster. Lists also have a faster function: nth.

concatenate concatenates copies of sequences together, producing a new sequence of a given type. The original sequences can be different types. concatenate looks like this:

(concatenate new-sequence-type sequences... )

new-sequence-type is a quoted symbol representing the type of the new sequence. For example, simple vectors use 'simple-vector and lists use 'list and strings use 'string.

make-sequence builds a sequence of a given type and length. Like elt, it is a general function (it calls faster, more type-specific functions underneath). It looks like this:

(make-sequence sequence-type length )

make-sequence has a keyword argument :initial-element which can be used to set the initial element of the sequence.

concatenate and make-sequence show the first examples of type symbols. We'll talk about types more later.

First off, most sequence-manipulative functions are either destructive or non-destructive. That is, either they modify or destroy the original sequence to achieve their goals (faster), or they make a copy of the sequence first. We'll show the non-destructive versions first.

Second, a great many sequence functions have three versions, the function, the -if version, and the -if-not version. For example, the count function also has count-if and count-if-not. The forms look like this:

(count object sequence keywords...) counts the number of times object appears in sequence.

(count-if test-predicate sequence keywords...) counts the number of times in which test-predicate (a function pointer) returns true for elements in sequence.

(count-if-not test-predicate sequence keywords...) counts the number of times in which test-predicate (a function pointer) returns false (nil) for elements in sequence.

Third, many such functions take a lot of optional keyword arguments. Before testing to see if an element is the one we're looking for, these functions give you a chance to "extract" the relevant item out of the element with an optional function passed in with the keyword argument :key. You can tell the system to scan backwards with :from-end t. You can tell the system to only scan from a certain location to another location in the sequence with the keywords :start and :end. There are other keywords as well.

Other functions which follow this pattern include: find (returns the first element matching the pattern) else nil, position (returns the index of the first element matching the pattern) else nil, remove (removes all the elements matching the pattern from a copy of the sequence), and substitute (replaces all the elements matching the pattern with some other element). substitute has an additional argument indicating the item to replace stuff with, thus its three versions substitute, substitute-if, and substitute-if-not start like this:

(substitute[-if[-not]] thing-to-replace-with rest-of-arguments-as-before...)

Another useful function, search, searches for the first index where one subsequence appears in another sequence. It takes the form:

(search subsequence sequence keywords...)

Keywords include :key, :test, :test-not, :from-end, :start1, :end1, :start2, :end2. Try them out and see what they do.

Many sequence functions have destructive counterparts which are faster but may modify the original sequence rather than making a copy first and modifying the copy.

There are no promises with destructive functions: they may or may not modify the original. They may or may not modify the original into the form you're hoping for. The only guarantee they make is that the value they return will be what you're hoping for. Thus you should only use them on data that you don't care about any more.

The destructive form of remove[-if[-not]] is delete[-if[-not]].

The destructive form of substitute[-if[-not]] is nsubstitute[-if[-not]].

The destructive form of reverse is nreverse.

sort is destructive. Its basic form looks like this:

(sort sequence predicate)

If a function call provides any extra arguments beyond those defined in the parameter list, the additional arguments are all placed in a list, which the &rest parameter is set to. Otherwise it is set to nil.

Though it's possible to have rest-parameters along with keyword parameters and optional parameters in the same function, don't do it. Ick.