portability framework

Conjure is meant to be portable across different Scheme implementations. That means we must maintain several source code flavours and, at the same time, making it easy to share common stuff. To that end, first of all we stick to R5RS and SRFIs whenever possible. In addition, we use Dorai Sitaram's scmxlate tool to keep shared and dialect-specific code separate.

How to port Conjure to your favourite Scheme
The Scheme 48 configuration language
Scmxlate in a nutshell
Conjure bootstrapping
Putting all together

How to port Conjure to your favourite Scheme

This is a quick summary of the steps needed to port Conjure to your favourite Scheme dialect, which we will call myscm (see the sections below for details).

1. Add myscm to scmxlate/dialects-supported.scm. 1. Write a scmxlate/myscm.scm file with definitions for the

following forms:

The Scheme 48 configuration language

As already noted, conjure uses a slightly extended subset of the Scheme48 configuration language. For those already knowing that, the differences are:

- There is only define-structure, no define-interface - The clauses allowed inside define-structure are:

For the unitiated, here's the overall structure of a define-structure clause:

(define-structure conjure.foo.bar (export qux baz
					  ((froboz fribnitz)
  (open scheme srfi-1 srfi-13 conjure.util.files)
  (dialect (guile (open ice-9.syncase))
           (scheme48 (open extended-ports)))
  (files (foo basic-bar)
         (foo bar)))

This defines the module conjure.foo.bar, which exports the identifiers qux and baz, as well as the macros froboz and fribnitz. The open clause means: this module is written in R5RS Scheme (scheme), plus additionally using srfi-1, srfi-13 and the conjure.util.files module. The dialect clause can be used for specifiying additional modules needed for the dialect-specific code for the module. Finally, the files clause tells us that this module consists of the files foo/basic-bar.scm and foo/bar.scm.

When porting conjure to another Scheme, a scmxlate-macro must be defined that maps the configuration language to something the target Scheme can understand. For example, the above would be translated into the following Guile module file:

(define-module (conjure foo bar)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-13)
  #:use-module (conjure util files)
  #:use-module (ice-9 syncase)

  #:export (qux baz)
  #:export-syntax (froboz fribnitz))

;; code for foo/basic-bar.scm and foo/bar.scm follows

Scmxlate in a nutshell

Here's a quick description of how scmxlate works1. For each dialect, we create a subdirectory that contains the implementation specific stuff. There you'll find for each module that needs extra tweaking for that dialect, a dialect-specific file named after the module's name. For instance, given this packages.scm:

(define-structure conjure.util.pregexp (export pregexp
  (open scheme srfi-1 srfi-23)
  (files (util pregexp)))

(define-structure conjure.util.named-args (export ((define/named-args)
  (open scheme srfi-23)
  (files (util named-args)))

(define-structure conjure.test.testeez (export ((testeez)
  (open scheme)
  (dialect (scheme48 (open extended-ports))
           (guile (open ice-9.syncase)))
  (files (test testeez)))

Suppose we're supporting Guile and Scheme 48, we'll have the following layout:

 |- guile.scm
 `- scheme48.scm
 |- packages.scm
 |- util
 |   |- pregexp.scm
 |   `- named-args.scm
 |- test
 |   `- testeez.scm
 `- dialects
     |- guile
     |   |- test.testeez.scm
     |   `- util.named-args.scm
     `- scheme48
         `- test.testeez.scm

The way conjure uses scmxlate is a bit special, as most things are done by the define-structure macro. In scmxlate parlance, packages.scm is the input file, that, after being processed by scmxlate according to the directives in guile.scm / scheme48.scm, will give rise to an output file, my-packages.scm 2, which is the result of translating the input file to the requested dialect. The define-structure macro defined in guile.scm / scheme48.scm is responsible for translating the files listed by the files clauses to their destination, which is dependent on the dialect.

For example, for Scheme 48, define-structure does little more than translate each file in the files clause seperatly (taking into account the dialect specific file for that module, if it exists) and filtering the non-scheme48 portions of dialect clauses out. For Guile, define-structure outputs translated versions of the files all into a single target file.

A variety of code transformations can be specified in the dialect files: renaming, redefinition, code insertion and arbitrary scheme code evaluation are available. For example, if foo.scm defines a procedure like, say3,

;; foo.scm
(define (file-newer a b)
  (if (and (exists? a) (exists? b))
      (> (file-timestamp a) (file-timestamp b))))

we would write the following guile-specific file providing a module declaration and a definition for file-timestamp (which is not available as such in Guile):

;; dialects/guile-foo.scm

;; scmxlate rename directive
(scmxlate-rename (exists? file-exists?))

(define (file-timestamp f) (stat:mtime (stat f)))

Or we could even redefine file-newer: if a (define file-newer ...) form appeared in the dialect-specific file, it would replace the one in foo.scm. scmxlate-rename is an example of directive specifiying a code transformation: you'll find many others in scmxlate's manual, or used in Conjure's source tree.

Conjure bootstrapping

To configure the source tree for a given dialect, you create a build directory, cd into it, and invoke configure from there. The scheme dialect of choice is specified with a --with-dialect option, e.g.

$ /path/to/src/configure --with-guile

The configure script creates two files in the build directory,

- scmxlate/scmxlate-conf.scm: scmxlate generic config for your dialect - bootstrap-pkg.sh: shell script in charge of invoking scmxlate,

populating your build directory with a dialect-specific version of Conjure's source code.

and invokes the later automatically4. It traverses the source tree, invokes scmxlate and copies (using scmxlate's directives) the output files to your build directory (instead of my-foo.scm, you end up with builddir/foo/foo.scm). The scmxlate program is located at scmxlate/scmxlate.scm, and in the same folder you'll find shared dialect-specific scmxlate configuration files, named dialect.scm (guile.scm, mzscheme.scm, etc.). These files are used during the invocation of scmxlate (which is accomplished by loading scmxlate.scm in bootstrap.sh). When this configuration ends, you'll have a copy of Conjure's code tunned to your implementation of choice in the build directory.

Putting all together

So, summing up, we start with an input file within a package in the source tree, say conjure/pkg/file.scm, and end up with a translated output file in the build directory, builddir/conjure/pkg/file.scm. The following graph depicts the files involved in the translation process, using Guile as a sample dialect:

conjure/pkg/file.scm -------------------     |
                                         \   V
scmxlate/scmxlate.scm ----------------------------> builddir/conjure/pkg/file.scm
conjure/pkg/dialects/guile-file.scm  -----
buildir/configure ===>    builddir/scmxlate/scmxlate-conf.scm

1. Please, refer to scmxlate's user manual for a detailed description.

2. As explained below, this default name can (and will) be changed.

3. Yeah, that's a silly example.

4. A second script, bootstrap-test.sh, is in charge of setting up Conjure's test suite. It is described in the testing framework description.