Currently supported platforms for development:
Windows is not supported because of issues building gems with C extensions (such as sassc).
For Mac (arm) see https://teaspoon-consulting.com/articles/archivesspace-on-the-m1.html.
When installing Java OpenJDK is strongly recommended. Other vendors may work, but OpenJDK is most extensively used and tested. It is highly recommended that you use Jabba to install Java (OpenJDK). This has proven to be a reliable way of resolving cross platform issues (looking at you Mac :/) that have occured via other means of installing Java.
Installing OpenJDK with jabba will look something like:
# assuming you have jabba installed jabba install email@example.com jabba use firstname.lastname@example.org jabba alias default email@example.com # [optional] make this the default java
If using Docker & Docker Compose install them following the official documentation:
Do not use system packages or any other unofficial source as these have been found to be inconsistent with standard Docker.
The recommended way of developing ArchivesSpace is to fork the repository and clone it locally.
Note: all commands in the following instructions assume you are in the root directory of your local fork unless otherwise specified.
This is an abridged reference for getting started with a limited explanation of the steps:
# Build images (required one time only for most use cases) docker-compose -f docker-compose-dev.yml build # Run MySQL and Solr in the background docker-compose -f docker-compose-dev.yml up --detach # Download the MySQL connector cd ./common/lib && wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.23/mysql-connector-java-8.0.23.jar && cd - # Download all application dependencies ./build/run bootstrap # OPTIONAL: load dev database gzip -dc ./build/mysql_db_fixtures/accessibility.sql.gz | mysql --host=127.0.0.1 --port=3306 -u root -p123456 archivesspace # Setup the development database ./build/run db:migrate # Clear out any existing Solr state (only needed after a database setup / restore after previous development) ./build/run solr:reset # Run the development servers supervisord -c supervisord/archivesspace.conf # OPTIONAL: Run a backend (api) test (for checking setup is correct) ./build/run backend:test -Dexample="User model"
ArchivesSpace development requires MySQL and Solr to be running. The easiest and recommended way to run them is using the Docker Compose configuration provided by ArchivesSpace.
Start by building the images. This creates a custom Solr image that includes ArchivesSpace’s configuration:
docker-compose -f docker-compose-dev.yml build
Note: you only need to run the above command once. You would only need to rerun this command if a) you delete the image and therefore need to recreate it, or b) you make a change to ArchivesSpace’s Solr configuration and therefore need to rebuild the image to include the updated configuration.
Run MySQL and Solr in the background:
docker-compose -f docker-compose-dev.yml up --detach
By using Docker Compose to run MySQL and Solr you are guaranteed to have the correct connection settings and don’t otherwise need to define connection settings for MySQL or Solr.
Verify that MySQL & Solr are running:
docker ps. It should list the running containers:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ec76bd09d73b mysql:8.0 "docker-entrypoint.s…" 8 hours ago Up 8 hours 33060/tcp, 0.0.0.0:3307->3306/tcp as_test_db 30574171530f archivesspace/solr:latest "docker-entrypoint.s…" 8 hours ago Up 8 hours 0.0.0.0:8984->8983/tcp as_test_solr d84a6a183bb0 archivesspace/solr:latest "docker-entrypoint.s…" 8 hours ago Up 8 hours 0.0.0.0:8983->8983/tcp as_dev_solr 7df930293875 mysql:8.0 "docker-entrypoint.s…" 8 hours ago Up 8 hours 0.0.0.0:3306->3306/tcp, 33060/tcp as_dev_db
To check the servers are online:
mysql -h 127.0.0.1 -u as -pas123 archivesspace
To stop and / or remove the servers:
docker-compose -f docker-compose-dev.yml stop # shutdowns the servers (data will be preserved) docker-compose -f docker-compose-dev.yml rm # deletes the containers (all data will be removed)
Advanced: running MySQL and Solr outside of Docker
You are not required to use Docker for MySQL and Solr. If you run them another way the default requirements are:
The defaults can be changed using environment variables located in the build file.
For licensing reasons the MySQL connector must be downloaded separately:
cd ./common/lib wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.23/mysql-connector-java-8.0.23.jar cd -
The bootstrap task:
Will bootstrap your development environment by downloading all
dependencies–JRuby, Gems, etc. This one command creates a fully
self-contained development environment where everything is downloaded
within the ArchivesSpace project
It is not necessary and generally incorrect to manually install JRuby & bundler etc. for ArchivesSpace (whether with a version manager or otherwise).
The self contained ArchivesSpace development environment typically does not interact with other J/Ruby environments you may have on your system (such as those managed by rbenv or similar).
This is the starting point for all ArchivesSpace development. You may need
to re-run this command after fetching updates, or when making changes to
Gemfiles or other dependencies such as those in the
Errors running bootstrap
[java] INFO: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 11+28 [java] Exiting [java] LoadError: no such file to load -- rails/commands [java] require at org/jruby/RubyKernel.java:974 [java] <main> at script/rails:8
./build/run backend:devserver ./build/run frontend:devserver ./build/run public:devserver ./build/run indexer
There have been various forms of the same
LoadError. It’s a transient error
that is resolved by rerunning bootstrap.
[java] org.jruby.Main -I uri:classloader://META-INF/jruby.home/lib/ruby/stdlib -r [java] ./siteconf20220407-5224-13f6qi7.rb extconf.rb [java] sh: /Library/Internet: No such file or directory [java] sh: line 0: exec: /Library/Internet: cannot execute: No such file or directory [java] [java] extconf failed, exit code 126
This has been seen on Mac platforms resulting from the installation method for Java. Installing the OpenJDK via Jabba has been effective in resolving this error.
Advanced: bootstrap & the build directory
Running bootstrap will download jars to the build directory, including:
Gems will be downloaded to:
The migrate task:
Will setup the development database, creating all of the tables etc. required by the application.
There is a task for resetting the database:
Which will first delete then migrate the database.
Advanced: Loading data fixtures into dev database
When loading a database into the development MySQL instance always ensure that ArchivesSpace
is not running. Stop ArchivesSpace if it is running. Run
./build/run solr:reset to
clear indexer state (a more thorough explanation of this step is described below).
If you are loading a database and MySQL has already been used for development you’ll want to drop and create an empty database first.
mysql -h 127.0.0.1 -u as -pas123 -e "DROP DATABASE archivesspace" mysql -h 127.0.0.1 -u as -pas123 -e "CREATE DATABASE IF NOT EXISTS archivesspace DEFAULT CHARACTER SET utf8mb4"
Note: you can skip the above step if MySQL was just started for the first time or any time you
have an empty ArchivesSpace (one where
db:migrate has not been run).
Assuming you have MySQL running and an empty
archivesspace database available you can proceed
gzip -dc ./build/mysql_db_fixtures/accessibility.sql.gz | mysql --host=127.0.0.1 --port=3306 -u root -p123456 archivesspace ./build/run db:migrate
Note: The above instructions should work out-of-the-box. If you want to use your own database and / or have configured MySQL differently then adjust the commands as needed.
After the restore
./build/run db:migrate is run to catch any migration updates. You can now
proceed to run the application dev servers, as described below, with data already
populated in ArchivesSpace.
The Solr reset task:
Will wipe out any existing Solr state. This is not required when setting
up for the first time, but is often required after a database reset (such as
after running the
./build/run db:nuke task).
More specifically what this does is submit a delete all request to Solr and empty
out the contents of the
./build/dev/indexer*_state directories, which is described
Use Supervisord for a simpler way of running the development servers with output for all servers sent to a single terminal window:
# run all of the services supervisord -c supervisord/archivesspace.conf # run in api mode (backend + indexer only) supervisord -c supervisord/api.conf # run just the backend (useful for trying out endpoints that don't require Solr) supervisord -c supervisord/backend.conf
ArchivesSpace is started with:
To stop supervisord:
Advanced: running the development servers directly
Supervisord is not required, or ideal for every situation. You can run the development servers directly via build tasks:
./build/run backend:devserver # This is the REST API ./build/run frontend:devserver # This is the staff user interface ./build/run public:devserver # This is the public user interface ./build/run indexer # This is the indexer (converts ASpace records to Solr Docs and ships to Solr)
These should be run in different terminal sessions and do not need to be run in a specific order or are all required.
An example use case for running a server directly is to use the pry debugger.
Advanced: debugging with pry
To debug with pry you cannot use supervisord to run the application devserver, however you can mix and match:
# run the backend and indexer with supervisord supervisord -c supervisord/api.conf # in a separate terminal run the frontend directly ./build/run frontend:devserver
binding.pry to set breakpoints in the code. This can also be used in views:
<% binding.pry %>. Using pry you can easily inspect the
in scope instance variables that are available.
Advanced: development servers and the build directory
Running the developments servers will create directories in
shared: background job files
Note: the folders will be created as they are needed, so they may not all be present at all times.
ArchivesSpace uses a combination of RSpec, integration and Selenium tests.
It’s also useful to be able to run the backend unit tests separately. To do this, run:
You can also run a single spec file with:
./build/run backend:test -Dspec="myfile_spec.rb"
By default the tests are configured to run using a separate MySQL & Solr from the development servers. This means that the development and test environments will not interfere with each other.
# run the backend / api tests ./build/run backend:test
You can also run a single spec file with:
./build/run backend:test -Dspec="myfile_spec.rb"
Or a single example with:
./build/run backend:test -Dexample="does something important"
There are specific instructions and requirements for the UI tests to work.
Advanced: tests and the build directory
Running the tests may create directories in
./build/test. These will be
the same as for the development servers as described above.
You can run the coverage reports using:
This runs all of the above tests in coverage mode and, when the run
finishes, produces a set of HTML reports within the
directory in your ArchivesSpace project directory.
If you are editing or adding source files that you intend to contribute via a pull request, you should make sure your changes conform to the layout and style rules by running:
Most errors can be auto-corrected by running:
./build/run rubocop -Dcorrect=true
When you have code ready to be reviewed, open a pull request to ask for it to be merged into the codebase.
To help make the review go smoothly, here are some general guidelines:
See: Building an Archivesspace Release for information on building a distribution.
See: Building an Archivesspace Release for information on building the documentation.