Stratus3D

Software Engineering, Web Development and 3D Graphics

Promises in Erlang

I recently took the University of Kent’s Concurrent Programming in Erlang course hoping that I would learn some new things about the Erlang VM. I have been programming in Erlang for 3 years so I was a little disappointed when the course only covered the more basic aspects of OTP and didn’t cover anything new to me. Overall the material in the course was a very well presented and it turned out to be a great way to refresh my knowledge of the Erlang concurrency primitives. There was a promise implementation that was briefly presented by Joe Armstrong that caught my attention, and that’s what I want to talk about here.

The code sample Joe presented stood out to me because it’s not the way things are normally done in Erlang. Promises are common in JavaScript, but with asynchronous message passing in Erlang there isn’t really a need for promises. If we have task that we want to run asynchronously in the background we can just spawn and link another process to do some work, and then wait for a message (or exit signal) in a receive block when we need the results. There are also things like Elixir’s Task module that can handle this and hide all the message passing. I was struck by how simple it is to implement simple promises in Erlang:

1
2
3
4
5
6
7
8
9
10
promise(Pid, Request) ->
    Tag = erlang:make_ref(),
    Pid ! {self(), Tag, Request},
    Tag.

yield(Tag) ->
    receive
        {Tag, Response} ->
            Response
    end.

Using these functions it’s easy to run a task in another process:

1
2
3
4
5
6
7
% Call the promise function with our function
Tag = promise(ExistingPid, fun() ->
        io:format("expensive task...", [])
    end).

% Receive the results when we need them
Result = yield(Tag).

All we need is the process ID of an existing worker function and we can pass the Pid and the function into the promise function. The promise function calls the erlang:make_ref/0 function returns a unique reference that we use in our response message. This ensures we don’t get things confused if a lot of promise messages are sent to the same process at the same time. Then the promise/2 function sends the function we provided to the worker process in a message, along with the reference. All we have to do is wait for a message back from the worker process in the yield/1 function. The yield/1 function blocks while waiting for a message back from the promise with the tag that is provided. The code running in the worker process isn’t show here, but it’s not hard to spawn a process that receives a function in a message, execute the function, and send a response message back to the caller Pid:

1
2
3
4
5
6
7
8
9
10
11
12
spawn_worker() ->
    spawn(fun worker_loop/0).

worker_loop() ->
    receive
      {Pid, Tag, RequestFun} ->
         % Do the work
         Result = RequestFun(),
         % Send the response back
         Pid ! {Tag, Result},
         worker_loop()
    end.

This all that is needed for the worker process. We just call spawn_worker/0 and that function spawns a process that executes worker_loop/0, which is a recursive function that will loop forever handling request messages by executing the function received and then sending the results back in a message. While this will work it isn’t ideal. In JavaScript we just pass the function into Promise object and get a promise back. We shouldn’t have to use an existing process to execute the function we passed in. The promise function should spawn a new process for us automatically. It’s easy to modify our promise function to do this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
promise(ExecutorFun) ->
    Tag = erlang:make_ref(),
    Self = self(),
    WorkerFun = fun() ->
        % Do the work

        Result = ExecutorFun(),

        % Send a message with the result back to the callers process
        Self ! {Tag, Result}
    end,
    spawn(WorkerFun),

    % Return the reference so we can use it in the yield function
    Tag.

All we have to do is spawn a new process inside the promise/1 function that executes the function we pass in and then sends the results back to the calling process in a message. Using the promise function is now even easier:

1
2
3
4
5
6
7
% Call the promise function with our function
Tag = promise(fun() ->
        io:format("expensive task...", [])
    end).

% Receive the results when we need them
Result = yield(Tag).

This is much better, but there is still room for improvement here. In JavaScript we have a catch function that we can add to our promise chains to handle exceptions that happen in the promise or previous then handlers. In Erlang there isn’t really a need for catch and then functions because we can pattern match on the result returned from the yield/1 function directly in the calling process. Currently if something goes wrong will our function is being executed by the promise the process will crash. We need to handle exceptions that may occur during the execution of the function so this doesn’t happen. We can do that by wrapping the function call in a try catch block and then sending a different message back to the calling process process based on the outcome:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
promise(ExecutorFun) ->
    Tag = erlang:make_ref(),
    Self = self(),
    WorkerFun = fun() ->
        % Do the work

        Response = try ExecutorFun() of
            Result ->
                % If everything goes as planned just return an `ok` tuple with the response
                {ok, Result}
            catch
                Error:Reason ->
                    % Otherwise return an error tuple with the exception type and reason
                    {error, Error, Reason}
        end,

        % Send a message with the result back to the callers process
        Self ! {Tag, Response}
    end,
    spawn(WorkerFun),

    % Return the reference so we can use it in the yield function
    Tag.

This is all that is needed. This promise function takes a function to execute just like the Promise object in JavaScript, spawns a process to run the function, and then sends a message back to the calling process after the function has returned or crashed. We’ve done all this in less than 30 lines of code. Using the new function that handles exceptions is easy. Here we create a promise that immediately crashes, and then we pattern match on the value returned from yield/1 in a case statement:

1
2
3
4
5
6
7
8
9
10
% Crash the promise process
Tag = promise(fun() -> exit(crash) end),
% Do some other things...
% Then get the results from the promise when we need it
case yield(Tag) of
    {ok, Result} ->
         io:format("Success!", []);
    {error, Error, Reason} ->
         io:format("Failed with ~p due to ~p", [Error, Reason])
end

We now have one block of code that is executed when things go as expected and another that is run when an exception occurs. As you can see there is no need to chain callback functions onto the promise because yield/1 blocks when we need to get the results and the current process will wait until the promise message has been received. There isn’t an equivalent in JavaScript for this. The closest we can get in JavaScript is chaining then and catch callbacks on a promise like this:

1
2
3
4
5
6
7
new Promise(function() {
           throw 'crash';
        }).then(function() {
            console.log("Success!");
        }).catch(function(reason) {
            console.log("Failed with " + reason);
        });

This is very different because the callbacks are executed asynchronously, whereas in the Erlang code above our yield function is synchronous. If we wanted asynchronous callbacks to be executed in the Erlang promise variation we would simply call them from inside the function we pass into the promise. Something like this:

1
2
3
4
5
6
ThenCallback = fun() -> io:format("Success!", []) end,

Tag = promise(fun() ->
        do_some_work(), % Do everything we need to
        ThenCallback() % Then execute our callback function
    end),

Conclusion

We’ve used spawn, ! (send), and receive to construct a promise function and a yield function in less than 30 lines of code. Our promise function handles executing the executor function in a another process, catching errors that may occur, and sending the appropriate message back to the calling process. The yield function handles receiving the response message and could also enforce a timeout if the promise took too long. This wouldn’t be possible without Erlang’s concurrency primitives. We don’t ever use promises in Erlang because of other things like gen_server and Elixir’s Task, but hopefully this blog post has shown some of the inherit flexible in Erlang’s concurrency primitives.

References

ZNC on WebFaction Shared Account

IRC is my preferred method of real-time communication. I love being able to spend time on IRC channels. With my schedule continuing to get busier and with less free time in general I’m not able to be on IRC all the time like I used to be. Being offline so much brings out the biggest weakness with IRC itself. Messages that are sent while you are offline are not queued. If someone sends you a message when you aren’t logged in the message will be lost forever. Only by staying logged in are you sure to get all your messages. That’s where bouncers come in. Bouncers stay logged into the IRC server for you and record all the messages you would have otherwise received had you been online. When you log in again the bouncer sends you all the messages you missed while you were away. ZNC is a popular open source bouncer that is easy to install and configure. In this blog post I’ll show how to get it running on a WebFaction hosting account.

Long Live IRC - xkcd comic

I use WebFaction’s shared hosting account. The price is unbeatable and the fact that it’s just a shared account and not a VM means it’s much faster than a VPS with equivalent specs. The best part is that you can install and run any software you want on your account as long as it doesn’t violate the terms of service. WebFaction terms of service allows things like ZNC so it’s the perfect place to install a bouncer. I highly recommend WebFaction. It’s only $10 a month for an account with 100GB of SSD storage, and it gets cheaper if you pay by the year. Here is an affiliate link if you are interested.

Installation

First login to your webfaction control panel and navigate to “Domains/Websites” > “Applications” and create a new application. For “App category” choose “Custom”. Give your app a name (I chose znc in this article). Then for “App type” choose “Custom App (listening on port)”. Then check the “Open port” checkbox and click save. You should be taken back to the applications index page. The applications page should show the application you just created along with the port number and IP address. Make a note of the IP and port, you’ll need them next.

SSH into your server and create a directory for the build:

mkdir -p $HOME/znc_build
cd $HOME/znc_build

Then download the ZNC source:

wget http://znc.in/releases/archive/znc-1.4.tar.gz
tar -xzf znc-1.4.tar.gz

I had to use version 1.4 since the compiler on my server didn’t have support for C++11 and ZNC version 1.4 was the last version to not require C++1. Apparently other shared WebFaction servers have support for C++11 so if yours does it would be best to use the latest version of ZNC.

Next configure your ZNC build to run the directory that was created for custom app you created. Mine was $HOME/webapps/znc so I ran:

./configure --prefix=$HOME/webapps/znc/

Next compile:

make && make install

If all goes as expected you should see this message:

******************************************************************
 ZNC was successfully installed.
 You can use '.../bin/znc --makeconf'
 to generate a config file.

 If you need help with using ZNC, please visit our wiki at:
   http://znc.in

Remove the build directory since ZNC is now installed:

cd ~
rm -rf ~/znc/

Before configuration let’s add the ZNC bin directory to our $PATH so it’s available in the shell. This will make things a lot easier in the long run. With my server the command I ran looked like this:

echo "export PATH=$HOME/webapps/znc/bin:$PATH" >> $HOME/.bashrc
# Then reload the bashrc
source ~/.bashrc

Configuration

Now we can configure ZNC. First generate the config file:

znc --makeconf

It will prompt you to enter a port number. Enter the port number you saw on the application page when you created the custom application in the WebFaction interface. After entering the port number follow the prompts to select your SSL and IPv settings. Then continue to follow the prompts and select the ZNC modules you want to enable. After that follow still more prompts to create a user. Make sure the user you create is an admin user. You will also be prompted to enter a “bind host”. Enter the IP address you say listed on the applications page in the WebFaction interface. You can create as many users as you want during setup. Continue through the rest of the prompts. The last one should ask if you want to start ZNC. Choose yes. Make a note of the configuration file location since it contains everything you’ll need to know to connect to ZNC.

Usage

Now that ZNC is running you should be able to connect to it. Try connecting to ZNC with your IRC client.

Irssi

Before connecting to your ZNC server you need to install the dispatch.pl script as described on this page and place it in your ~/.irssi/scripts/autorun/ directory. This plugin allows you to use /znc in Irssi to send commands to ZNC. Once you have the file in place run /script load autorun/dispatch to load it. Then add your ZNC server:

/server add -net myserver -auto -ssl <hostname or ip> <port> <username>:<password>

Replace myserver with the name you want to give to your ZNC server. The name will be used when you connect to ZNC. Be sure to enter the same IP and port you got from the WebFaction interface just like with ZNC. Then connect by running /connect myserver. Once you are connected use /znc to issue ZNC commands. /znc help should be enough to get you started.

WeeChat

Connecting to ZNC is even simpler in WeeChat. Just add the server and tell it to autoconnect:

/server add myserver <hostname or ip>/<port> -ssl -username=<username> -password=<password>
/set irc.server.myserver.autoconnect on

Again, replace myserver with whatever name you want to give your server. You may also need to turn ssl_verify off if you aren’t using it:

/set irc.server.myserver.ssl_verify off

Then you should be able to connect by running /connect myserver or restarting WeeChat. Once again you can use /znc to issue ZNC commands once you are connected.

Conclusion

Getting ZNC setup is just the beginning. There is still more that needs to be done in order to have ZNC recording everything you care about while you are offline. That’s too much for this blog post though. Checkout the documentation on ZNC website or run /znc help to learn more about configuring your ZNC account.

As you have seen ZNC is simple to install and configure. With WebFaction it’s also very economical to run. I’m really glad I set up ZNC as it has really helped me get more out of IRC. Hopefully this blog post has inspired you to try out ZNC or another bouncer. As always, if you have any questions or suggestions don’t hesitate to send me an email.

Resources

Lua Version Management With Asdf-lua

I’ve written asdf in a past blog post but I haven’t covered asdf-lua in depth yet. In this blog post I’ll talk about the advantages of asdf-lua over existing version managers and show how it’s used.

Lua version management is a pretty small niche. The most popular Lua version manager on GitHub has only 91 stars. asdf-lua isn’t a self-contained Lua version manager but rather a plugin for asdf, and extendable version manager with support for over a dozen languages. Unless you are already using asdf chances are you have several version managers already installed. Rather than installing yet another version manager you can use asdf to manage all the languages you rely on in development. But asdf will work fine with your existing version managers too so you don’t need to worry about uninstalling anything to use asdf-lua. While it might seem like asdf-lua is more complex than other version managers it’s really not. asdf-lua itself is only two short Bash scripts and both asdf and asdf-lua are easy to install and configure as we will see.

Installation

If you don’t have asdf installed already you will need to install it. It’s easy:

# Clone the repo, currently the latest release is 0.2.1
git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.2.1

# Then add two lines to your bashrc or bash_profile if you are on OSX
echo -e '\n. $HOME/.asdf/asdf.sh' >> ~/.bashrc
echo -e '\n. $HOME/.asdf/completions/asdf.bash' >> ~/.bashrc

That’s it. After reloading your .bashrc you should be able to run asdf in your shell and see the help printed out. Next install asdf-lua:

asdf plugin-add lua https://github.com/Stratus3D/asdf-lua.git

You will also need a C compiler to compile Lua. Refer to the readme for instructions on getting gcc installed on your OS.

Now that we have asdf-lua and it’s dependencies installed using it to install Lua versions is easy.

Usage

To see the Lua versions that are available to install run:

asdf list-all lua

This command will print out a list of all the Lua versions. Find one you want to install and run the installation command. I’ve chosen to install lua 5.3.2:

asdf install lua 5.3.2

After Lua is compiled you can run asdf list lua and you should see the Lua version you just installed. Now that you have a Lua version installed you just need to tell asdf when to use it.

Specifying Lua Versions

asdf looks for a .tool-version file in the current directory and reads it to figure out which version should be used when lua or luac is invoked in that directory. If the file doesn’t exist in the current directory asdf navigates up to the parent directory and checks for it again. This is repeated until a .tool-version file is found on the root directory is reached. Typically you want to set the Lua version on a per-project basis, but you may also want to set a global Lua version so it is available everywhere.

Setting the Lua version for a project

To set a Lua version for a project cd into the project’s root directory and create the tool version file:

echo "lua 5.3.2" > .tool-version

You should now be able to run lua -v in the directory to see the Lua version being set. You can also use asdf current lua to see the Lua version as well as the path to the .tool-version file that set it. Rather than creating a .tool-version file manually for every project you can the asdf local command to create it for you:

asdf local lua 5.3.2

Setting a global Lua version

Since asdf is relying on the presence of a .tool-version file in the current directory or one of the parent directories all you need to do to set a global Lua version is to create .tool-version file in your home directory. There is a command that makes this easy:

asdf global lua 5.3.2

This command works just like asdf local only it creates a .tool-version file in your home directory. You can see what it’s written by running less ~/.tool-version. You should now be able to use Lua in any directory under your home directory.

Conclusion

asdf-lua is comparable to other version managers in many ways. The real benefit to asdf-lua is asdf itself. With asdf there is no need to learn the quirks of yet another version manager just to be able to install multiple versions of Lua. The commands I’ve shown here are the exact same for asdf-ruby, asdf-elixir, and all the other asdf plugins.

Feedback is always welcome so if you have any questions about asdf-lua or this blog post feel free to contact me. I’m always looking for ways of improving asdf-lua and asdf so don’t hesitate to contact me if you have any suggestions.

References