Stratus3D

Software Engineering, Web Development and 3D Graphics

Erlang Project Setup With Erlang.mk.

I have been using erlang.mk on all of the new projects I have started over the last 9 months. Most of my Erlang applications are structured the same and so the project setup process is nearly identical for each one. I might eventually write a bash script to automate the setup new Erlang projects. In the meantime, I will document the setup process here. Feel free to contact me if you have better methods for project setup. I will gladly link to other useful resources relating to Erlang project setup.

In this post I assume you have Git and Erlang installed. I will be ignoring Erlang versions throughout this post, but I am using Erlang R16B01. Everything in the post should work with Erlang 17, but some small changes may be required.

1. Erlang.mk and Initial Project Setup

Create a directory for your new project (my will be stored in the code directory):

$ cd code
$ mkdir erlang_app

Then clone down erlang.mk in a separate directory:

$ cd ..
$ git clone ssh://git@github.com/ninenines/erlang.mk

Next copy the erlang.mk file out of the erlang.mk directory and into the new project, then enter the directory containing your project:

$ cp erlang.mk/erlang.mk erlang_app/
$ cd erlang_app

Next we will use erlang.mk to generate a Makefile and source directory with Erlang files necessary to create a minimal Erlang release:

$ make -f erlang.mk bootstrap

A Makefile and a src directory now exist. Inside the src directory you will see three files:

erlang_app_app.erl  erlang_app.app.src  erlang_app_sup.erl

These erlang_app_app contains the application behavior callbacks and erlang_app_sup contains the callbacks for the supervisor behaviour. The erlang_app.app.src is a basic application resource file containing the application specification (more on that later).

Next check if our empty application compiles. With the Makefile generated by erlang.mk the default target is all. The all target has the build target as prerequisite. The build target compiles all the files stored in src. Run the default target to compile the code:

$ make

The application is now compiled. An ebin directory now exists. Inside are two beam files and the application resource file:

$ ls ebin/
erlang_app.app  erlang_app_app.beam  erlang_app_sup.beam

In addition to the default target, erlang.mk provides numerous other targets that are useful:

$ make clean # Removes all the beam files and the ebin directory
$ make tests # Run all the CT or EUnit tests
$ make deps # Download and compile dependencies
$ make dialyze # Run dialyzer
$ make rel # Build a release if relx is present (more on that next)

We still don’t have an easy way to generate a release. There are still benefits to being able to build releases right now, before the application does anything useful:

  • Releases make it easy to start and stop your application for demos and integration tests performed during development.
  • It allows you to verify that a change to your codebase or the structure of your application didn’t break the build and release process.
  • In production your code will be inside a release. By building releases during development you be able to verify you application code runs correctly inside a release. This helps iron out issues with paths and such.

2. Relx and Erlang Releases

Before we install Relx we need to create a Relx configuration file. This file is used by Relx when building releases. Create a file named relx.config in the project root and add the follow to the file:

% This specifies the release name and version and the applications that are part of the release (this would be our application along with any dependencies it might have)
{release, {erlang_app, "0.1.0"}, [erlang_app]}.
% The location of the `sys.config`, the file containing the application configuration (this will be copied into the release)
{sys_config, "./config/sys.config"}.
% Use a complete start script
{extended_start_script, true}.
% Don't include the source
{include_src, false}.

After saving relx.config create a directory named config and inside it a file named sys.config with the following contents:

[{erlang_app, []}].

This file is used to store all the configuration data for our erlang_app application. The second value in the tuple is proplist of config values. For now it can be left empty.

Next we will need to edit our application resource file. The application source file is in the src directory and ends in .app.src. Mine is named erlang_app.app.src. It contains an Erlang term that contains application metadata like version number, description, etc. There should be a line in the file that contains a tuple starting with modules. Mine looks like this:

{modules, [erlang_app_app, erlang_app_sup]},

Relx will look at this tuple to determine what modules need to be included in the release. Make sure all the modules that make up your application are listed here. Otherwise they won’t be present in the releases. While you have the file open you should also fill in the other fields, although many of them are not required.

Next we will need to install Relx itself. There are two ways to install it. You can build and install it manually or you can let erlang.mk build and install the latest version of it for you. The benefit to installing it manually is that you can control which version you use.

Install Relx Manually

Clone down Relx into a directory outside the project:

$ cd ..
$ git clone ssh://git@github.com/erlware/relx.git

Then enter the relx directory and build the relx binary:

$ cd relx
$ make

Make should build the relx binary. The relx binary should exist in the directory. Go back to directory containing the application you are building and copy the relx binary into the project (this commands assumes the Relx repository and the directory containing your application are in the same parent directory):

$ cp ../relx/relx .

You now have the relx binary in your root of your Erlang application. Next make the relx binary executable:

$ chmod a+x relx

Automatically Install Relx with erlang.mk

Erlang.mk will look for a relx.config file in the project root. Since we have already created it, all we need to do is run the release target:

$ make rel

Erlang.mk will download and install Relx, placing the relx binary in your project root.

Run Relx

No matter what method you used to install Relx you should now have a relx binary in your project root. Now you can run the release target by running:

$ make rel

You may see an error like this when you run the command:

 GEN    distclean-relx-rel
 ===> Starting relx build process ...
 ===> Resolving OTP Applications from directories:
           /usr/local/stow/erlang_R16B01/lib/erlang/lib
           No goals specified for this release ~n
           make: *** [relx-rel] Error 127

This is due to the fact that the application is not compiled. The solution is to compile and then build the release:

$ make
$ make rel

You can now start the application in the release by running:

$ cd _rel/erlang_app
$ ./bin/erlang_app start

You should now have an application that can be compiled to bytecode and packaged in a release. You can stop here. However, the current setup doesn’t lend itself to rapid development. When we make a change to the source code we must build a new release and run it manually to test our changes. While this will work it will make for a very slow development. With Sync, changes to source code can be automatically reloaded in the erl console. This makes for a very fast development workflow.

3. Faster Workflow with Sync

In order for the sync commands to be available in the console the Sync project must be stored inside a directory that is part of your ERL_LIBS environmental variable. This could be anywhere on your system. For my project, I will be using the ~/code directory as the ERL_LIBS path. To install Sync run:

$ cd ~/code
$ git clone ssh://git@github.com/rustyio/sync.git

Then build Sync by running:

$ cd sync
$ make

The project will compile and a bunch of .beam files will appear in the ebin directory. Next, set $ERL_LIBS by adding a line to your .bashrc:

export ERL_LIBS=$HOME/code

You could also run this directly in your terminal if you don’t want to add it to your .bashrc. If you updated your .bashrc you will need to reload it by running source ~/.bashrc. Check that ERL_LIBS is set before continuing:

$ echo $ERL_LIBS
/home/stratus3d/code

After verifying that ERL_LIBS is set correctly return to the Erlang application you are working on and open up the erl console:

$ cd ~/code/erlang_app
$ erl

The sync module should be available in the Erlang console:

$ erl
Erlang R16B01 (erts-5.10.2)...

Eshell V5.10.2  (abort with ^G)
1> sync:module_info().
[{exports,[{start,0}... }]
2>

To make Sync listen for changes and reload modules in the console run:

1> sync:go().
Starting Sync (Automatic Code Compiler / Reloader)
Scanning source files...
ok

Sync will now watch for changes to your Erlang source files. When a file changes it will be recompiled and reloaded into the console by Sync. Other useful Sync commands include:

1> sync:stop(). % Stop sync
2> sync:pause(). % Pause sync. Sync will stop recompiling and reloading modules, run sync:go() to resume

Another thing you can do to make spinning up a console session easier is to create a target in your Makefile to automate the starting of Sync. Add this to your Makefile just above the line that includes erlang.mk:

console:
    erl -s sync go

Remember that the actions in Makefile targets are indented with tabs and not spaces. Now when you run the target an Erlang console will be opened and Sync will be started automatically:

$ make console
erl -s sync go
Erlang R16B01 (erts-5.10.2) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V5.10.2  (abort with ^G)
Starting Sync (Automatic Code Compiler / Reloader)
1> Scanning source files...

Sync is a new addition to my workflow but so far it has been very helpful in speeding up the development process.

Summary

Hopefully this post has been useful to you. I have found this method to be one of the simplest ways to get a new Erlang project up and running. With the addition of Sync, development iterations can be sped up significantly. There were several things I wanted to cover in this post but wasn’t able to. The obvious next step would be to start writing tests. Erlang.mk makes running tests relatively trivial, although re-running tests after changes is still done manually. Another thing is version control with Git. I didn’t cover creating a .gitignore for the project. Perhaps I will cover these in a future post.

Resources:

Update (7/9/2015)

Originally the second line of the relx.config was tuple with an empty list as it’s third item.

{release, {erlang_app, "0.1.0"}, []}.

This is wrong. The list at the end of the tuple should contain all applications that are part of the release. At minimum we need to include the app we are releasing like this:

{release, {erlang_app, "0.1.0"}, [erlang_app]}.

Update 2 (4/26/2016)

A couple people have emailed me questions about an error they get when running make rel:

$ make rel
 DEPEND erlang_app.d
 ERLC   erlang_app_app.erl erlang_app_sup.erl
 APP    erlang_app.app.src
===> Starting relx build process ...
===> Resolving OTP Applications from directories:
          /Users/user/projects/erlang_app/ebin
          ...
===> Failed to solve release:
 Dependency erlang_app is specified as a dependency but is not reachable by the system.

This error message is very generic and can be caused by a number of things in the erlang_app.app.src or relx.config so it’s impossible to know what the exact cause is. Here are a couple things I would check:

  • Check all the modules names in the list of modules and make sure all the modules in the list exist in your project (that’s the “{modules, […]},” line). If a module is missing or spelled incorrect building the release will fail.
  • Check all the other places module or process names in listed in your erlang_app.app.src file. (under the list of modules or registered processes, or the name of the application). Any naming discrepancies will generate this error.
  • Check the version (vsn) listed in your erlang_app.app.src file and make sure it corresponds with the version in your relx.config. If the version numbers do not match relx won’t be able to find the application.
  • Check all the other values used in relx.config. There are several other things in your relx.config that may cause this error.

For me the issues usually turns out to be a misspelled module name or an incorrect version number. Unfortunately the cryptic error message makes debugging difficult.