Overview

Stability level

Current stability level for Multithreaded Javascript Tasks:

Alpha
Beta 1
Beta 2
Stable
Visit here to report an issue or request a feature enhancement.

Back To Top

Introduction

Another important feature introduced in JXcore is multithreading. In a single-threaded project setTimeout() and setInterval() methods wait in a separate thread, but the scheduled methods are still executed under the main application thread. Since the main thread eventually handles every single task, the application may lost its responsiveness under a load or queued heavy tasks. Even if the application works well in a testing environment, it could start slowing down under a massive load (increased number of clients).

Back To Top

Memory management

Since node is based on V8 engine, it also inherits it’s defaults and limitations.

Currently, the default V8 memory limit is 512 MB on 32-bit systems and 1 GB on 64-bit systems. According to the documentation, this can be raised to a maximum of 1 GB and 1.7 GB respectively.

But one of the most important factors about running multithreaded tasks in JXcore is the fact that the main thread as well as every subthread uses its own V8 heap space, so the V8 memory limits applies to each of them separately! For example, an application which runs on four threads can hold up to 5 x 1.7 GB of memory (1 for the main thread and 4 for the subthreads)!

And when there are no currently active tasks, JXcore subthreads force an automatic V8 heap cleanup – each of them separately on its own!

Moreover, the queue mechanism for the subthreads uses separate memory blocks than V8 engine. It means that it occupies a different place in the memory, hence the queues do not fall under the limitations mentioned above.

Back To Top

How to run multithreaded code?

Two ways

There are two ways of executing your JavaScript code in multiple threads with JXcore.

The first and the easiest is just to use mt or mt-keep option in the command line for jx:

> jx mt-keep:4 easy1.js

You can read more about this in the next article The easiest way.

The second one is by using jxcore.tasks object. It requires you to implement all the multithreaded logic by yourself. Then you run the application without any jx’s options:

> jx tasks_way.js

Please refer to The “tasks” way for more information.

The main difference between those two methods is, that with mt/mt-keep approach you have to do absolutely nothing to run the application multithreaded. However, the same code is running separately for each thread and you don’t really have any control to change the thread’s job, once the application started.

With the second (the “tasks”) approach it’s all about adding jobs to the queue of the thread pool. Tasks start and finish, you can always add more tasks in the runtime. In this model you always have the main thread and the subthreads. You can also get notified, when all task are completed. See the API for reference.

Back To Top

The easiest way

Consider this piece of code (easy1.js):

var http = require('http');

http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/html'});
    res.end("Hello World from thread " + process.threadId);
}).listen(1337, '127.0.0.1');

console.log("Server running at http://127.0.0.1:1337 on thread number " + process.threadId);

The most simple way to run this application in multiple threads is to use a command line:

> jx mt-keep:4 easy1.js

The mt-keep:4 parameter means that it will run the code in 4 threads. It will also keep them all alive after the listen() method returns.

Now the console output shows us:

mt_the_easiest_way1

We just started the http server in 4 separate threads, but all of them are sharing the same server’s listener handle for processing incoming requests.

If we now walk with the browser to:

http://127.0.0.1:1337

our application will assign a random idle thread to this request, and we may see the output like:

There is also another usage, with mt parameter:

> jx mt easy1.js

which does not keep the subthread alive. This means that its execution finishes as soon as the running code completes in a synchronous way.

Both mt, and mt-keep parameters may be used with or without specifying the number of threads, like:

> jx mt:10 easy1.js
> jx mt-keep:5 easy1.js

In case if we don’t supply the number of threads, as in example below:

> jx mt-keep easy1.js

JXcore by default will create 2 threads in the pool.

Back To Top

The “tasks” way

The main idea in this approach is to define the tasks, which you will then add to the thread pool for execution. For this, you can use any of the methods described in the API section.

The task can do anything. See the simplest example:

    var method = function () {
        console.log("this is message from thread no", process.threadId);
    };

    jxcore.tasks.addTask(method);

Another one:

    var method = {};
    method.define = function () {
        require("./hello");
        console.log("http server started on thread", process.threadId);
        process.keepAlive();
    };

    jxcore.tasks.runOnce(method);

Note that this particular example does nothing but running hello.js file in multiple threads. The same result can be obtained by using mt:keep option as follows:

> jx mt:keep hello.js

We have written a blog post showing How to turn your existing application into a multithreaded one with a few lines of code.

Back To Top

Tutorials

API test

Let’s consider the following apitest.js example.

We can define a method, which we would like to run in separate thread, and it would return a result to the main thread:

var method = function (obj) {
    console.log("We are in a subthread now.");
    console.log("Argument passed from the main thread: ", obj);
    console.log("subthread index: ", process.threadId);
    console.log("am i really a subthread?: ", process.subThread);
    return { someResult: "some result" };
};

Now, using JXcore, you can easily do this:

jxcore.tasks.addTask(method, {count: 1000}, function (result) {
    // this block of code executes back in the main thread,
    // after method() completes.
    console.log("This is result from subthreaded method:", result);
});

What happens here is that we move the execution of method() to another thread, so the main thread is not blocked until method() completes.

This is the output from the application:

mt_api_test1

Back To Top

Basics

Another sample we would like to show is basics.js file.

Here we have a different method:

var method = function (obj) {
    var t = 0;
    for (var i = 0; i < obj.count; i++) {
        for (var z = 0; z < 100000; z++) {
            t += z % 2;
        }
    }
    return {total: t};
};

We also add the task a bit later to the thread pool, not to block the main thread at this specific moment. This is why we use setInterval() method:

setInterval(function () {
    start = process.hrtime();

    jxcore.tasks.addTask(method, {count: 1000}, function (result) {
        console.log("result", result.total);
    });
}, 1000);

Finally, we are attaching to the event, which fires after all tasks have been processed:

// this event occurs when all the added tasks have been processed
jxcore.tasks.on('emptyQueue', function () {
    console.log('total time spent', process.hrtime(start), process.memoryUsage());
});

Back To Top

PostgreSQL

In this tutorial we want to show how you can use a PostgreSQL database with JXcore multithreaded tasking module.

For small number of tasks something like this would suffice:

var method = function () {

    var conn = require('util').format("tcp://%s:%s@%s:%s/%s", user, pwd, host, port, db);

    var pg = require('pg');
    var client = new pg.Client(conn);
    client.connect();

    var query = client.query("select count(*) from jxcore");

    query.on("row", function (row) {
        console.log("Result: ", row);
        process.release()
    });

    process.keepAlive();
};

Let’s than say, that our computer’s processor has 4 cores, so the jxcore.tasks module would utilize 3 of them. After we execute the task with runOnce():

jxcore.tasks.runOnce(method);

the task will be executed for each of those 3 subthreads separately, which means, that 3 database connections will be created.
This is exactly what we wanted – we have just created multithreaded access to the database.

But what if we would like to have additional tasks doing this job. For example:

for (var a = 1; a <= 200; a++) {
    jxcore.tasks.addTask(method, a);
}

then sooner or later we can hit some serious problems, starting from:

Uncaught Error: connect ECONNRESET
// or
Uncaught error: sorry, too many clients already

This can happen, because each of the 200 tasks was creating its own database connection. This approach is not efficient because the pg module will create 200 processes (one per each connection). Surely we cannot proceed this way.

Definitely we need another solution. It would still be enough to have one database connection per subthread, and not for every task.

For this kind of scenario we introduced define & logic approach in jxcore. Please refer to the documentation for more information.

The idea here is to create database connections once per subthread inside define() method:

var task = {};

task.define = function () {

    var conn = require('util').format("tcp://%s:%s@%s:%s/%s", user, pwd, host, port, db);

    var pg = require('pg');
    var client = new pg.Client(conn);
    client.connect();
};

The second part is logic() method. We are going to use static client variable, created inside define(), to make all calls to the database:

task.logic = function(id) {

    var query = client.query("select count(*) from jxcore");

    query.on("row", function (row) {
        process.release()
    });

    process.keepAlive();
};

Now we can safely add as many tasks as we want, i.e. 1,000,000. Obviously, this will take some time to process, but it will work!

for (var a = 1; a <= 1000000; a++) {
    jxcore.tasks.addTask(task, a);
}

As a final note, please be adviced, that the same task object, which we just have created with define & logic methods, can be used with runOnce():

jxcore.tasks.runOnce(task);

but there will only be 3 tasks, instead of a million.

Back To Top