This is the final part of 4 part Reactor Pattern series. In Reactor Pattern Part 1 : Applications with Blocking I/O, I went through issues faced by a single threaded application to scale to handle more requests pre box. In Reactor Pattern Part 2 : Applications with Non-Blocking I/O I went through what Reactor Pattern is and how it solved the Blocking IO issues and mentioned call back issue due to async code. Reactor Pattern Part 3 – Promises to solve callback hell talked about callback code issues in detail and how promise library can solve some of the issues.

Recap – Part 3 Conclusion

Async or Non-blocking IO introduces new challenges on how applications should to be structured and how async call backs can be abstracted away using promises like library. We need Non-blocking IO application since, sequential blocking IO applications are not scalable. So Non-Blocking IO or Asynchronous code is not a desired feature but a necessary evil to achieve scalability.

Finally, we should question assumption that Non-blocking IO and Asynchronous code are clubbed together and one comes with other. Is it possible to get the best of both the worlds, i.e. Sequential code and Non-Blocking IO scalability. In fact, I think there is an option based on Fibers which can provide best of both the worlds.

An interesting article on Why Events Are A Bad Idea makes sequential control flow arguments in more depth.

I will cover Fibers and how they achieve both Non-Blocking IO and Sequential codebase and demonstrate it with an expressjs resful service in the this blog.

Pre-emptive and co-operative multitasking is a good place to begin understanding Fibers.

Pre-emptive Multitasking and Co-Operative Multitasking

Pre-emptive Multitasking :–

In computing, preemption is the act of temporarily interrupting a task being carried out by a computer system, without requiring its cooperation, and with the intention of resuming the task at a later time. Such a change is known as a context switch. It is normally carried out by a privileged task or part of the system known as a preemptive scheduler, which has the power to preempt, or interrupt, and later resume, other tasks in the system. – from Wikipedia

Linux Scheduler (privileged task) pre-empts process tasks without its co-operation. The disadvantage of preemptive multitasking is that the OS may make a context switch at an inappropriate time.

Co-Operative Multitasking :–

Early multitasking systems used applications that voluntarily ceded time to one another. This approach, which was eventually supported by many computer operating systems, is known today as cooperative multitasking. – from Wikipedia

Co-Operative Multitasking relies on the threads relinquishing control once they are at a stopping point. The disadvantage of co-operative multitasking is that a poorly written application can blocking the entire system. Real-time embedded systems are often implemented using Co-Operative Multitasking paradigm to get real time performance.

What are Fibers?

Fibers are lightweight threads (also called green threads) which are process or application level concepts and don’t correspond to OS threads. They provide thread like execution flow. While OS threads are pre-emtively scheduled, programmer can use fibers to co-opratively multitask. Fibers are conceptually similar to coroutines .i.e. execution can be suspended and resumed programmatically.

Lets see an example of how fibers work

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

var Fiber = require('fibers');

var log_sequence_counter = 1;

function sleep(task, milliseconds) {
    var fiber = Fiber.current;
    setTimeout(function() {
      console.log(log_sequence_counter++ + task + ' callback');
        fiber.run();
    }, milliseconds);
    console.log(log_sequence_counter++ + task + ' thread/fiber suspended');
    Fiber.yield();
    console.log(log_sequence_counter++ + task + ' thread/fiber resumed');
}

var task1 = function() {
    console.log(log_sequence_counter++ + ' task 1 waiting for sleep to end ');
    sleep(" task 1",1000);
    console.log(log_sequence_counter++ + ' task 1 got back from sleep');
}

var task2 = function() {
    console.log(log_sequence_counter++ + ' task 2 waiting for sleep to end ');
    sleep(" task 2", 1000);
    console.log(log_sequence_counter++ + ' task 2 got back from sleep');
}

Fiber(task1).run();
Fiber(task2).run();
console.log( log_sequence_counter++ + ' main execution flow');

In the above example, you can notice that

• Fibers are created using Fiber() function and the task function is executed using run method.
• Pattern used to suspend and resume a Fiber. This pattern will be reused while making Non-Blocking IO calls.
• Within a fiber thread Fiber.current returns current executing fiber.
• Fiber.yield suspends execution of current thread i.e. voluntarily relinquish control. In other words it allows another fiber thread execute co-operatively.
• task1 and task2 functions don’t have callbacks or promises and is sequential code.
• task1 and task2 functions don’t know about fibers and developers can read/write these functions as sequential code.
• Fibers provide co-operative multi-tasking capability

1
2
3
4
5
6
7
8
9
10
11
12

$ node fibersExample.js
1 task 1 waiting for sleep to end
2 task 1 thread/fiber suspended
3 task 2 waiting for sleep to end
4 task 2 thread/fiber suspended
5 main execution flow
6 task 1 callback
7 task 1 thread/fiber resumed
8 task 1 got back from sleep
9 task 2 callback
10 task 2 thread/fiber resumed
11 task 2 got back from sleep

The output of fibersExample.js shows the other of execution with sequence numbers. Even though NodeJS is single threaded application, the above code demonstrates – how multiple tasks are run with-out blocking each other.

Fibers – In ExpressJS Restful Service

Lets look at an ExpressJS restful service example, to keep code simple, I have not included exception handling which can be done using try – catch blocks as we do other languages. It provides three service methods

1. /google :– Makes a http get call to google and returns html response from google to client.
2. /user/:fb_id :– Return User JSON for given facebook id.
3. /user/:fb_id/events :– Returns User and User’s Events for a given facebook id.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

var express = require('express');
var fibersMiddleWare = require('./fib-middleware');
var request = require("./fib-request");
var fib_redis = require("./fib-redis");
var redis_client = fib_redis.init(require("redis-url").connect());

/*------------------------------------Models----------------------------------*/

var User = {};
User.get = function(id){
              var user_json = redis_client.get("user:" + id);
              return JSON.parse(user_json);
          };

var Event = {};
Event.getUserEvents = function(user_id){
                        var user_json = redis_client.mget("user:" + id + ":events");
                        return JSON.parse(user_json);
                    };

/*----------------------------------------------------------------------------*/

var app = express();
app.use(fibersMiddleWare.runInFiber);

app.get('/google', function(req, res){
  var google_response_body = request.get('http://google.com')
  res.send(google_response_body);
});

app.get('/users/:fb_id',function(req, res){
                  var user = User.get(req.params.fb_id)
                  res.setHeader('Content-Type', 'application/json');
                  res.send(200,JSON.stringify(user));
                });

app.get('/users/:fb_id/events',function(req, res){
                  var user = User.get(req.params.fb_id)
                  var events = Event.getUserEvents(user.id);
                  var response = {'user' : user, 'events' : events};

                  res.setHeader('Content-Type', 'application/json');
                  res.send(200,JSON.stringify(response));
                });

var server = app.listen(3000, function() {
    console.log('Listening on port %d', server.address().port);
});

In the above example, we have used few custom/wrapper libraries ‘fib-middleware’, ‘fib-request’ and ‘fib-redis’. They are simple extensions of simple fibers example above.

You can notice that

• Fibers are setup as middleware using app.use(fibersMiddleWare.runInFiber)
• Controller and Model methods are Synchronous and Sequential Code.
• Http get Request and Datastore (redis_client) operations are also Synchronous.
• Other than sequential code there is nothing special happening with-in Server.js code above.
• Code demonstrates that Non-Blocking IO code can be synchronous and sequential

Lets looks at the libraries that server.js depends on

1
2
3
4
5
6
7
8
9

var Fiber = require("fibers");

function fiberMiddleWare(req,resp,next){
  Fiber(function(){
      next();
  }).run();
}

exports.runInFiber = fiberMiddleWare

ExpressJS middleware is an implementation intercepting filters pattern which can be used to perform any processing before and after passing the request to controller method.

fib-middleware.js is a simple mechanism to process all http requests within a fiber context. It is similar to Fiber(task1).run() in the fibersExample.js above.

Custom Wrapper Libraries to provide Fibers Support

fib-request.js and fib-redis.js are wrappers which provide Fibers support and are not required if original library (redis.js or request.js) support Fibers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

var request = require('request');
var Fiber = require("fibers");

function get(url){
  var error,response,body;
  var fiber = Fiber.current;
  request.get(url,
                    function(err, resp, b){
                      error = err;
                      response = resp;
                      body = b;
                      fiber.run()
                    });
  Fiber.yield();
  return body;
}

exports.get = get;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

var Fiber = require('fibers');
var conn;

function get(key){
  var error, value;
  var fiber = Fiber.current;
  conn.get(key, function(err,val){
      error = err;
      value = val;
      fiber.run();
    });
    Fiber.yield();
    return value;
}
function init(connection){
   conn = connection;
   return { 'get':get};
}

exports.init = init;

As you can see, fib-request.js and fib-redis.js follow code pattern similar to the fibers example of sleep where thread is suspended and resumed to make the function synchronous.

The above sample ExpressJS application shows that it is possible to write sequential Non-Blocking I/O using Fibers. Though I have used NodeJS in the sample codebase. It is not restricted to NodeJS, the same applies to other languages which support Fibers (light-weight threads) like Ruby, Python etc.

Libraries Supporting Fibers

Most application codebase can be divided into two major parts

1. Business or functional part :– All business and function logic of application is written in this part of codebase by project team developers. Most of developers time is spend working on this part of application.
2. Framework or Library part :– In all projects we end-up using several libraries like mvc framework, database drivers etc. which are third part libraries developed by developers outside team. These usually are used across projects and developers usually spend every little time modifying or changing this part.

In the express sample above

1. Server.js belongs to Business application part.
2. ExpressJS, Fibers.js, redis.js, fib-middleware.js, fib-request and fib-redis.js belong to framework or library part. i.e. once implemented by library developers or project team, it can be re-used across projects.

As shown in the above code, creating fiber based wrapper libraries for NodeJS libraries is pretty simple.

Recently, library developers started supporting Promise Library, which was not the case before. As fibers gain wider developer acceptance, library developers would also support fibers.

Asynchronous Vs Parallel

Most of the examples we have seen till now (this and previous blogs on reactor pattern) has been an instance of synchronous code converted to asynchronous code to make it non-blocking. While this is a common scenario and in normal projects it covers 90 to 95% of scenarios in NodeJS applications. But there are occasional requirement which require multiple parallel requests to be made and wait for all the responses to get back. This is a genuine case where parallel or async processing is required and should be allowed to work asynchronously in usual NodeJS Async pattern with help Promise Library (Q.all()).

Performance/Scalability

Siege based Performance testing of async call-back based code and sequential fibers code (shown above) showed similar results on my MacBook Pro.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

$ siege -r 10 -c 100 http://localhost:3000/users/11265765672

Transactions:             1000 hits
Availability:           100.00 %
Elapsed time:            10.06 secs
Data transferred:         0.20 MB
Response time:                0.01 secs
Transaction rate:        99.40 trans/sec
Throughput:               0.02 MB/sec
Concurrency:              0.66
Successful transactions:        1000
Failed transactions:             0
Longest transaction:          0.05
Shortest transaction:         0.00

1
2
3
4
5
6
7
8
9
10
11
12
13
14

$ siege -r 10 -c 100 http://localhost:3000/users/11265765672

Transactions:             1000 hits
Availability:           100.00 %
Elapsed time:             9.09 secs
Data transferred:         0.20 MB
Response time:                0.01 secs
Transaction rate:       110.01 trans/sec
Throughput:               0.02 MB/sec
Concurrency:              0.92
Successful transactions:        1000
Failed transactions:             0
Longest transaction:          0.06
Shortest transaction:         0.00

Conclusion

With Lightweight thread (Fibers) we can do Co-Operative multi-tasking which voluntarily relinquish control and resume processing. If application framework and libraries support Fibers or library wrappers created as shown in above example, functional and application logic can be written in synchronous style. Presence of Fibers, async code can be abstracted into Framework and libraries from functional code. As functional code tend to be larger than wrapper code that might be required, this will substantially reduce application complexity.