Lua Lanes - Comparison
Keywords: Lua, Library, Multithreading, Threads, Rocks

## Comparisons to other threading kits

A short comparison of Lanes with other existing Lua multithreading kits.

    =============
      Lua Lanes
    =============

    With the introduction of Lindas (Jun-2008), Lua Lanes simplifies its API while
    simultaneously adding more power and speed.

    Pros:
        - regular Lua 5.1/5.2 module
        - completely separate Lua states, one per OS thread
        - message passing, or shared data using Lindas
        - no application level locking, ever
        - scales well, up to 1000's of threads
        - priorities (-2..+2) for launched threads
        - threads are cancellable (with complete cleanup)
        - timeouts on all pending operations
        - thread contents are given as regular Lua functions; syntax checked early on,
          syntax highlighting works
        - standard libraries opened to subthreads can be granually selected
        - fast stack-to-stack copies, via hidden "keeper states". No serialization needed.
        - protects calls to 'require', allowing wide compatibility with existing
          modules (and all, with minor changes)

    Cons:
        - requires OS threads
        - currently 1:1 mapping to OS threads (limits scalability and maybe performance)
        - not utilizing network parallelism (all threads on one CPU)

    Sample:
    <<
      lanes = require "lanes".configure()

      local function calculate(a,b,c)
        if not a then
          error "sample error; propagated to main lane when reading results"
        end
        return a+b+c
      end

      local h1= lanes.gen("base", calculate)(1,2,3)
      local h2= lanes.gen("base", calculate)(10,20,30)
      local h3= lanes.gen("base", calculate)(100,200,300)

      print( h1[1], h2[1], h3[1] )     -- pends for the results, or propagates error
    <<

    ===========
      luaproc    (by Skyrme, Rodriguez and Ierusalimschy)
    ===========

    http://www.inf.puc-rio.br/~roberto/docs/ry08-05.pdf

    The PDF seems to be an authorative voyage into how Lua could handle multithreading,
    in a multicore "separate universes" manner (not like what coroutines already do).

    Pros:
        - Tackles both multicore and network parallelism
        - M:N relationship to kernel threads (one kernel thread runs multiple luaprocs)
        - Simple (so they say)
        - Lua author (Roberto) included
        - Can be used also without _any_ OS threading support (works like Rings, then)

    Cons:
        - Data passing for "strings, number, or booleans" only
          "More complex types must be encoded in some form"
          (serializing data is slower than the stack-to-stack copies used by i.e. Lanes)
          (yet, serializing allows for network parallelism)
        - Message passing is synchronous (only). The sender will wait until the
          receiver has taken the message.

    ==================
      Lua coroutines    (by Lua authors)
    ==================

    http://www.lua.org/manual/5.1/manual.html#2.11
    http://lua-users.org/wiki/CoroutinesTutorial

    Lua coroutines is an integral part of Lua 5 itself. It is listed here, since
    it should be the _first_ multitasking mechanism to consider. It can also be
    used together with Lanes, or the other mechanisms listed below.

    Coroutines are very usable in provider/consumer situations, allowing you to
    queue Lua functions on an as-needed dataflow basis with each other.

    Pros:
        - works with plain Lua (no extensions)
        - works on any platform
        - lightweight (no OS level threading or locking involved)

    Cons:
        - co-operative, meaning your code will need to decide, who gets to run
        - does not utilize multiple CPUs/cores

    Sample:

        ..TBD: sample code doing the "child" "parent" output here (see below)..

    =============
      LuaThread     (by Diego Nehab)
    =============

    http://www.cs.princeton.edu/~diego/professional/luathread/

    LuaThread provides thread creation, mutexes, condition variables, and inter-
    thread queues to the Lua scripts. It takes a C-kind of approach, where Lua
    globals are shared by the threads running, and need therefore to be guarded
    against multithreading conflicts.

    Whether this is exactly what you want, or whether a more loosely implemented
    multithreading (s.a. Lanes) would be better, is up to you. One can argue that
    a loose implementation is easier for the developer, since no application level
    lockings need to be considered.

    Pros:
        - no marshalling overhead, since threads share the same Lua state

    Cons:
        - requires a modified Lua core
        - application level locking required

    Sample:
    <<
      local function flood(output, word)
        while 1 do
            output:lock()
            io.write(word, ", ")
            output:unlock()
        end
      end

      local output = thread.newmutex()
      thread.newthread(flood, {output, "child"})
      flood(output, "parent")
    <<

    =============
      Lua Rings     (by Roberto Ierusalimschy & Tomás Guisasola)
    =============

    http://www.keplerproject.org/rings/

    ".. library which provides a way to create new Lua states from within Lua.
    It also offers a simple way to communicate between the creator (master) and
    the created (slave) states."

    ".. master can execute code in any of its slaves but each slave only has
    access to its master (or its own slaves)."

    Rings offers separate Lua states, but no multithreading. This makes it simple,
    but it won't use more than one CPU core. Other differences include:

        - opens all Lua standard libraries for subthreads
          (Lanes opens the needed ones)

        - marshalls numbers, strings, booleans, userdata
          (Lanes also marshalls tables, functions, upvalues, ..)

        - "remotedostring" allows executing code in the master state
          (Lanes does _not_ allow subthreads to trouble/modify master automatically,
          to allow effective sandboxing. The same can be achieved by sending code
          between the threads, but master needs to explicitly allow this = receive
          a function and execute it)

        - offers "Stable, a very simple API to manage a shared table at the master
          state"
          (Lanes 2008 offers keeper tables)

    Pros:
        - "offers Stable, a very simple API to manage a shared table at the master
        state"

    Cons:
        - thread contents defined as strings, not Lua source as such; does not
          give syntax check at file parsing, does not allow syntax highlight

    Sample:
    <<
      require"rings"
      S = rings.new ()

      data = { 12, 13, 14, }
      print (S:dostring ([[
      aux = {}
      for i, v in ipairs (arg) do
    	table.insert (aux, 1, v)
      end
      return unpack (aux)]], unpack (data))) -- true, 14, 13, 12

      S:close ()
    <<

    ==========================
      Helper Threads Toolkit    (by Javier Guerra G.)
    ==========================

    http://luaforge.net/projects/helper-threads/

    "Provides a consistent framework to write non-blocking C libraries, with a Lua
    interface for starting tasks and managing the Futures, Queues and Threads."

    This seems like a companion of the "occasional threading" model (see below);
    Lua side is kept clear of multithreading, while C side can be "spawn" off to
    do things on the background.

    Pros:
        - provides an "update" mechanism, allowing the (C) thread and controlling
          Lua to interact during execution of the thread
        - ...

    Cons:
        - thread is "only for C code and it can't touch or access the Lua state",
          in other words there is no Lua-side multithreading concept (by design)

    ========================
      Occasional threading      (by Russ Cox)
    ========================

    http://lua-users.org/lists/lua-l/2006-11/msg00368.html

    ".. able to have certain C calls run in parallel with the [Lua] VM, but
    otherwise keep the VM single-threaded."

    That pretty much says it all.

    Pros:
        - simple, yet providing for the "occasional" need to run really multithreaded
        - can be made into a loadable module (says the message)

    Cons:
        - only for occasional usage, the programming paradigm is still essentially
          singlethreaded (by definition)
        - not a real project, just a message on the Lua list (but a good one!)

    =================
      ConcurrentLua
    =================

    http://concurrentlua.luaforge.net/index.html

    ConcurrentLua is based on the Lua model for concurrency, namely coroutines, and
    extends this model by providing message-passing primitives.

    ".. implementation of the share-nothing asynchronous message-passing model"

    ".. process can check its mailbox for new messages at any time, and if there
    are any, they can be read in the order of arrival."

    ".. processes in the system are implemented with Lua coroutines"

    ".. still based on the cooperative multithreading model that Lua uses"

    Recent, released on 21 June 2008.

    Pros:
        - From ground up designed for distributed computing (multiple computers,
        not only CPU cores)
        - Does not require a pre-emptive operating system

    Cons:
        - Serialization must degrade raw performance in one-computer scenarios
          (vs. stack-to-stack copying ala Lanes)
        - Depends on LuaSocket and Copas modules.
        - Each thread has a single mailbox tied to it (vs. separating threads and
          connection resources)

For feedback, questions and suggestions:

- Lanes @ GitHub
- the maintainer
- the lua mailing list