Asdf Tip: Global Programs

Every so often I install a program via gem, npm or some other language specific package manager and want it to be available globally on my system. Npm has the -g flag but this doesn’t help when I change Node versions. The same goes for Ruby. If I switch to a different version of Ruby I can’t use the command I installed in the earlier version.

Promises in Erlang

I recently took the University of Kent’s Concurrent Programming in Erlang course hoping that I would learn some new things about the Erlang VM. I have been programming in Erlang for 3 years so I was a little disappointed when the course only covered the more basic aspects of OTP and didn’t cover anything new to me. Overall the material in the course was a very well presented and it turned out to be a great way to refresh my knowledge of the Erlang concurrency primitives. There was a promise implementation that was briefly presented by Joe Armstrong that caught my attention, and that’s what I want to talk about here.

The code sample Joe presented stood out to me because it’s not the way things are normally done in Erlang. Promises are common in JavaScript, but with asynchronous message passing in Erlang there isn’t really a need for promises. If we have task that we want to run asynchronously in the background we can just spawn and link another process to do some work, and then wait for a message (or exit signal) in a receive block when we need the results. There are also things like Elixir’s Task module that can handle this and hide all the message passing. I was struck by how simple it is to implement simple promises in Erlang:

promise(Pid, Request) ->
    Tag = erlang:make_ref(),
    Pid ! {self(), Tag, Request},
    Tag.

yield(Tag) ->
    receive
        {Tag, Response} ->
            Response
    end.

Using these functions it’s easy to run a task in another process:

% Call the promise function with our function
Tag = promise(ExistingPid, fun() ->
        io:format("expensive task...", [])
    end).

% Receive the results when we need them
Result = yield(Tag).

All we need is the process ID of an existing worker function and we can pass the Pid and the function into the promise function. The promise function calls the erlang:make_ref/0 function returns a unique reference that we use in our response message. This ensures we don’t get things confused if a lot of promise messages are sent to the same process at the same time. Then the promise/2 function sends the function we provided to the worker process in a message, along with the reference. All we have to do is wait for a message back from the worker process in the yield/1 function. The yield/1 function blocks while waiting for a message back from the promise with the tag that is provided. The code running in the worker process isn’t show here, but it’s not hard to spawn a process that receives a function in a message, execute the function, and send a response message back to the caller Pid:

spawn_worker() ->
    spawn(fun worker_loop/0).

worker_loop() ->
    receive
      {Pid, Tag, RequestFun} ->
         % Do the work
         Result = RequestFun(),
         % Send the response back
         Pid ! {Tag, Result},
         worker_loop()
    end.

This all that is needed for the worker process. We just call spawn_worker/0 and that function spawns a process that executes worker_loop/0, which is a recursive function that will loop forever handling request messages by executing the function received and then sending the results back in a message. While this will work it isn’t ideal. In JavaScript we just pass the function into Promise object and get a promise back. We shouldn’t have to use an existing process to execute the function we passed in. The promise function should spawn a new process for us automatically. It’s easy to modify our promise function to do this:

promise(ExecutorFun) ->
    Tag = erlang:make_ref(),
    Self = self(),
    WorkerFun = fun() ->
        % Do the work

        Result = ExecutorFun(),

        % Send a message with the result back to the callers process
        Self ! {Tag, Result}
    end,
    spawn(WorkerFun),

    % Return the reference so we can use it in the yield function
    Tag.

All we have to do is spawn a new process inside the promise/1 function that executes the function we pass in and then sends the results back to the calling process in a message. Using the promise function is now even easier:

% Call the promise function with our function
Tag = promise(fun() ->
        io:format("expensive task...", [])
    end).

% Receive the results when we need them
Result = yield(Tag).

This is much better, but there is still room for improvement here. In JavaScript we have a catch function that we can add to our promise chains to handle exceptions that happen in the promise or previous then handlers. In Erlang there isn’t really a need for catch and then functions because we can pattern match on the result returned from the yield/1 function directly in the calling process. Currently if something goes wrong will our function is being executed by the promise the process will crash. We need to handle exceptions that may occur during the execution of the function so this doesn’t happen. We can do that by wrapping the function call in a try catch block and then sending a different message back to the calling process process based on the outcome:

promise(ExecutorFun) ->
    Tag = erlang:make_ref(),
    Self = self(),
    WorkerFun = fun() ->
        % Do the work

        Response = try ExecutorFun() of
            Result ->
                % If everything goes as planned just return an `ok` tuple with the response
                {ok, Result}
            catch
                Error:Reason ->
                    % Otherwise return an error tuple with the exception type and reason
                    {error, Error, Reason}
        end,

        % Send a message with the result back to the callers process
        Self ! {Tag, Response}
    end,
    spawn(WorkerFun),

    % Return the reference so we can use it in the yield function
    Tag.

This is all that is needed. This promise function takes a function to execute just like the Promise object in JavaScript, spawns a process to run the function, and then sends a message back to the calling process after the function has returned or crashed. We’ve done all this in less than 30 lines of code. Using the new function that handles exceptions is easy. Here we create a promise that immediately crashes, and then we pattern match on the value returned from yield/1 in a case statement:

% Crash the promise process
Tag = promise(fun() -> exit(crash) end),
% Do some other things...
% Then get the results from the promise when we need it
case yield(Tag) of
    {ok, Result} ->
         io:format("Success!", []);
    {error, Error, Reason} ->
         io:format("Failed with ~p due to ~p", [Error, Reason])
end

We now have one block of code that is executed when things go as expected and another that is run when an exception occurs. As you can see there is no need to chain callback functions onto the promise because yield/1 blocks when we need to get the results and the current process will wait until the promise message has been received. There isn’t an equivalent in JavaScript for this. The closest we can get in JavaScript is chaining then and catch callbacks on a promise like this:

new Promise(function() {
           throw 'crash';
        }).then(function() {
            console.log("Success!");
        }).catch(function(reason) {
            console.log("Failed with " + reason);
        });

This is very different because the callbacks are executed asynchronously, whereas in the Erlang code above our yield function is synchronous. If we wanted asynchronous callbacks to be executed in the Erlang promise variation we would simply call them from inside the function we pass into the promise. Something like this:

ThenCallback = fun() -> io:format("Success!", []) end,

Tag = promise(fun() ->
        do_some_work(), % Do everything we need to
        ThenCallback() % Then execute our callback function
    end),

Conclusion

We’ve used spawn, ! (send), and receive to construct a promise function and a yield function in less than 30 lines of code. Our promise function handles executing the executor function in a another process, catching errors that may occur, and sending the appropriate message back to the calling process. The yield function handles receiving the response message and could also enforce a timeout if the promise took too long. This wouldn’t be possible without Erlang’s concurrency primitives. We don’t ever use promises in Erlang because of other things like gen_server and Elixir’s Task, but hopefully this blog post has shown some of the inherit flexible in Erlang’s concurrency primitives.

References

https://hexdocs.pm/elixir/Task.html
https://www.futurelearn.com/courses/concurrent-programming-erlang
https://www.futurelearn.com/courses/concurrent-programming-erlang/1/steps/173396

ZNC on WebFaction Shared Account

IRC is my preferred method of real-time communication. I love being able to spend time on IRC channels. With my schedule continuing to get busier and with less free time in general I’m not able to be on IRC all the time like I used to be. Being offline so much brings out the biggest weakness with IRC itself. Messages that are sent while you are offline are not queued. If someone sends you a message when you aren’t logged in the message will be lost forever. Only by staying logged in are you sure to get all your messages. That’s where bouncers come in. Bouncers stay logged into the IRC server for you and record all the messages you would have otherwise received had you been online. When you log in again the bouncer sends you all the messages you missed while you were away. ZNC is a popular open source bouncer that is easy to install and configure. In this blog post I’ll show how to get it running on a WebFaction hosting account.

I use WebFaction’s shared hosting account. The price is unbeatable and the fact that it’s just a shared account and not a VM means it’s much faster than a VPS with equivalent specs. The best part is that you can install and run any software you want on your account as long as it doesn’t violate the terms of service. WebFaction terms of service allows things like ZNC so it’s the perfect place to install a bouncer. I highly recommend WebFaction. It’s only $10 a month for an account with 100GB of SSD storage, and it gets cheaper if you pay by the year. Here is an affiliate link if you are interested.

Installation

First login to your webfaction control panel and navigate to “Domains/Websites” > “Applications” and create a new application. For “App category” choose “Custom”. Give your app a name (I chose znc in this article). Then for “App type” choose “Custom App (listening on port)”. Then check the “Open port” checkbox and click save. You should be taken back to the applications index page. The applications page should show the application you just created along with the port number and IP address. Make a note of the IP and port, you’ll need them next.

SSH into your server and create a directory for the build:

mkdir -p $HOME/znc_build
cd $HOME/znc_build

Then download the ZNC source and extract the files out of the archive:

wget http://znc.in/releases/archive/znc-1.4.tar.gz
tar -xzf znc-1.4.tar.gz
cd znc-1.4

I had to use version 1.4 since the compiler on my server didn’t have support for C++11 and ZNC version 1.4 was the last version to not require C++1. Apparently other shared WebFaction servers have support for C++11 so if yours does it would be best to use the latest version of ZNC.

Next configure your ZNC build to run the directory that was created for custom app you created. Mine was $HOME/webapps/znc so I ran:

./configure --prefix=$HOME/webapps/znc/

Next compile:

make && make install

If all goes as expected you should see this message:

******************************************************************
 ZNC was successfully installed.
 You can use '.../bin/znc --makeconf'
 to generate a config file.

 If you need help with using ZNC, please visit our wiki at:
   http://znc.in

Remove the build directory since ZNC is now installed:

cd ~
rm -rf ~/znc_build

Before configuration let’s add the ZNC bin directory to our $PATH so it’s available in the shell. This will make things a lot easier in the long run. With my server the command I ran looked like this:

echo "export PATH=$HOME/webapps/znc/bin:$PATH" >> $HOME/.bashrc
# Then reload the bashrc
source ~/.bashrc

Configuration

Now we can configure ZNC. First generate the config file:

znc --makeconf

It will prompt you to enter a port number. Enter the port number you saw on the application page when you created the custom application in the WebFaction interface. After entering the port number follow the prompts to select your SSL and IPv settings. Then continue to follow the prompts and select the ZNC modules you want to enable. After that follow still more prompts to create a user. Make sure the user you create is an admin user. You will also be prompted to enter a “bind host”. Enter the IP address you saw listed on the applications page in the WebFaction interface. You can create as many users as you want during setup. Continue through the rest of the prompts. The last one should ask if you want to start ZNC. Choose yes. Make a note of the configuration file location since it contains everything you’ll need to know to connect to ZNC.

Usage

Now that ZNC is running you should be able to connect to it. Try connecting to ZNC with your IRC client.

###Irssi

Before connecting to your ZNC server you need to install the dispatch.pl script as described on this page and place it in your ~/.irssi/scripts/autorun/ directory. This plugin allows you to use /znc in Irssi to send commands to ZNC. Once you have the file in place run /script load autorun/dispatch to load it. Then add your ZNC server:

/server add -net myserver -auto -ssl <hostname or ip> <port> <username>:<password>

Replace myserver with the name you want to give to your ZNC server. The name will be used when you connect to ZNC. Be sure to enter the same IP and port you got from the WebFaction interface just like with ZNC. Then connect by running /connect myserver. Once you are connected use /znc to issue ZNC commands. /znc help should be enough to get you started.

###WeeChat

Connecting to ZNC is even simpler in WeeChat. Just add the server and tell it to autoconnect:

/server add myserver <hostname or ip>/<port> -ssl -username=<username> -password=<password>
/set irc.server.myserver.autoconnect on

Again, replace myserver with whatever name you want to give your server. You may also need to turn ssl_verify off if you aren’t using it:

/set irc.server.myserver.ssl_verify off

Then you should be able to connect by running /connect myserver or restarting WeeChat. Once again you can use /znc to issue ZNC commands once you are connected.

Conclusion

Getting ZNC setup is just the beginning. There is still more that needs to be done in order to have ZNC recording everything you care about while you are offline. That’s too much for this blog post though. Checkout the documentation on ZNC website or run /znc help to learn more about configuring your ZNC account.

As you have seen ZNC is simple to install and configure. With WebFaction it’s also very economical to run. I’m really glad I set up ZNC as it has really helped me get more out of IRC. Hopefully this blog post has inspired you to try out ZNC or another bouncer. As always, if you have any questions or suggestions don’t hesitate to send me an email.

Resources

https://en.wikipedia.org/wiki/Internet_Relay_Chat
http://wiki.znc.in/ZNC
https://community.webfaction.com/questions/6763/znc-violation-of-webfaction-eula-and-can-it-be-done
http://wiki.znc.in/Irssi
http://wiki.znc.in/WeeChat
https://weechat.org/files/doc/devel/weechat_quickstart.en.html
https://weechat.org/files/doc/weechat_faq.en.html

← Older Blog Archives Newer →

About Me

I'm a software engineer working at Rent from Florida. I'm passionate about building reliable systems with Erlang and Elixir. I'm a maintainer of asdf and the creator of eFlambé.

Stratus3D

A blog on software engineering by Trevor Brown

Asdf Tip: Global Programs

Promises in Erlang

Conclusion

References

ZNC on WebFaction Shared Account

Installation

Configuration

Usage

Conclusion

Resources