Embedded Scripting Languages

Table of Contents

1 Embedded Scripting Languages

1.1 Overview

Reasonable Features of Embedded Scripting Languages

This section lsits some reasonable features that a language designed to be embedded should have:

  • Ligthweight - Small footprint, small size and memory requirements.
  • Scripting Engine as a library:
    • Must be available as C or C++ library which exposes the interpreter API allowing the client code to evaluate scripting code from strings or files and also to retrieve objects from the virtual machine memory.
  • Sandboxing or capability limitations
    • A reasonable requirement for an embedded scripting language is limiting the capabilities and APIs which can be used possibly allowing the execution of non-trusted scripts. Example: the Javascript engines of most web browsers do not allow the script to interact with file systems or operating system APIs.
  • Permissive License for static linking
  • Use Cases
    • DSL - Domain Specific Languages
    • Game Engines
    • Configuration files => Data Description language.
    • User content
    • Allow application runtime changes without recompilation.
    • User extension without modification of source code.

Examples of embedded scripting languages usage and use-cases

  • TCL - Tool Command Language => Used by many EDA - Electronic Design Automation Sofware in electronic engineering.
  • JavaScript => Used in Web Browsers, which are written mostly in C++, for controlling user interaction, animation and etc.
  • TinyScheme => Used by GNU GIMP drawing application ans Apple's MacOSX sandbox configuration.
  • Lua language => Used by many game engines and also by applications such as: Nginx web server; Linux Conky; Geany Editor; NMap editor and so on.
  • Squirrel Language => Used in game engines
  • Python
    • => Python is used as embedded scripting language by GDB - GNU Debugger ; WinDBG - Windows Debugger; IDA - Debugger for Reverse Engineering.
    • Disadvantages: Python has a large footprint; it was not designed as an embedded scripting language and it is not possible to forbid the interpreter from calling file system and process creation APIs.
  • AutoLisp [proprietary] => Lisp-like language used in Autocad.
  • SQL (Structured Query Language) => Many databases are implemented in C or C++ uses SQL as scripting language and domain specific language (DSL) for querying and storing data. Example: SQLite, Postgres SQL, …
  • Emacs Lisp => Emacs core, including the LISP engine/interpreter, is written in C and other parts are written in Emacs Lisp. This lisp dialect allows extending Emacs at with custom extensions (plugins) and also modifying the application behavior at runtime.
  • VBA - Visual Basic for Application [proprietary] => Used in Microsft Office Suite, specially in Microsft Excel.

Considerations for choosing embedded scripting languages

  • Small footprint and small overhead
  • Stability of C or C++ API
  • C++ API bindings
  • Permissive license for static linking or option of dual license for static linking.
  • Documentation and of C or C++ binding APIs, examples about binding code
  • Garbage collector implementation
  • Heavy duty computations with significant overhead should be performed on the C++-side, specially loops.
  • JIT - Just-In-Time compiler => can increase the performance by translating bytecodes into machine code.
    • Example: Lua JIT, Javascript V8 engine used by Chrome browser.
  • Features for running untrusted scripts or configuration:
    • Ability for restricting capabilities such as creating process, accessing the file system and so on.
    • Non-turing complete => better for configuration
  • Familiarity of the targe audience
    • Lua is pervasive in Games
    • TCL is pervasive on Electronic Design Automation Software
    • Javascript is widely used on the Web, NodeJS and also as embedded scripting language of web browsers.

Further Reading

1.2 Embedded Scripting Languages Selection

Selection of embedded scripting languages and engines available as libraries:

Non categorized:

  • TCL - Tool Command Language
  • mruby (Embeddable Ruby implementation)
    • License: BSD
    • Implementation: C
    • Syntax type: Ruby
  • Wren
    • License: MIT
    • Implementation: C
    • Syntax Type: N/A
  • Jinx
    • License: MIT
    • Implementation: C++17
    • Syntax Type: N/A
  • Gravity
    • License: -
    • Implementation: C
    • Syntax type: Apple's SWIFT language
  • DaScript
    • License: -
    • Implementation: C++14
    • Syntax type: Akin to Python

Lua or similar to Lua (mimics Lua syntax)

  • Lua
    • License: MIT
    • Implementation: C
    • Syntax Type: syntax inspired by scheme.
  • LuaJIT (Lua with JIT - Just-in-Time compiler which translates bytecodes to native machine code for better performance.)
    • License: MIT
    • Implementation: C
    • Syntax type: lua
  • GameMonkey Script
    • License: MIT
    • Implementation: C++
    • Syntax type: Akin to Lua
  • Squirrel
    • License: MIT
    • Implementation: C++
    • Syntax type: Lua-like
    • Note: Despite be implemented in C++, does not expose a C++ API, it exposes a C API.
  • Squilu (Squirrel fork)
    • License: MIT
    • Implementation: C++
    • Syntax type: Lua-like

Similar to C++ (syntax that mimics C++)

  • AngelScript (Note: statically typed)
    • License: Zlib
    • Implementation: C++
    • Syntax type: C++-like

Smiliar to Javascript or subset of Javascript (ECMAScript)

Similar to Lisp or Scheme

  • ECL - Embeddable Common Lisp
    • License: LGPL - 2
    • Implementation: C
    • Syntax type: Lisp, Common Lisp
  • TinyScheme
    • License: BSD
    • Implementation: C
    • Syntax tyope: scheme, lisp
    • Note: Used in GNU GIMP as scripting language and Apple MacOSX's sandbox as configuration language.
  • S7 Scheme (Variant of TinyScheme)
    • License: BSD
    • Implementation: C
    • Syntax type: C
  • Chibi Scheme
    • Implementation: C
    • Syntax type: Scheme, Lisp
  • Janet Language
    • License: MIT
    • Implementation: C
    • Syntax type: Clojure, Lisp
  • ArkScript
    • License: MPL license
    • Implementation: C
    • Syntax type: Lisp-like, more clojure-like

License obligations and requirements:

  • LGPL allows dynamically linking of closed source applications, but static linking requires source code disclosure and release under the same license. Some LGPL libraries, such as QT, allows static linking via commercial license.
  • MIT, BSD, APACHE and so on => Add a copy of the license; Give credit.

1.3 MuParser - Math expression parser

MuParser is a non-turing complete embedded scripting engine for evaluating math expressions.

Web Site:

Repository:

Conan Reference:

Sample Project

File: CMakeLists.txt

cmake_minimum_required(VERSION 3.9)
project(cppexperiments)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_VERBOSE_MAKEFILE ON)

# ============= Conan Bootstrap =============================#

# Download automatically, you can also just copy the conan.cmake file
if(NOT EXISTS "${CMAKE_BINARY_DIR}/conan.cmake")
   message(STATUS "Downloading conan.cmake from https://github.com/conan-io/cmake-conan")
   file(DOWNLOAD "https://github.com/conan-io/cmake-conan/raw/v0.13/conan.cmake"
                 "${CMAKE_BINARY_DIR}/conan.cmake")
endif()

include(${CMAKE_BINARY_DIR}/conan.cmake)

# Possible values "default" and "llvm8"
set(CONAN_PROFILE default)

conan_cmake_run(REQUIRES
                muparser/2.2.6@conan/stable
                BASIC_SETUP
                BUILD missing)

 #======= Targets Settings ===============+#

add_executable(muparser1_formula muparser1_formula.cpp)
target_link_libraries(muparser1_formula muparser)

File: muparser1_formula.cpp

#include <iostream>
#include <string>
#include <cmath>

#include <muParser.h>

// Defined for UNIX-like: Linux, Mac OSX, QNX, ...
#if __unix__
  #include <unistd.h>
#endif

bool isTTY_terminal()
{
   #if __unix__
      return ::ttyname(STDIN_FILENO) != nullptr;
   #else
      return false;
   #endif
}

double future_value(double PV, double rate, double nper)
{
    return PV * std::pow(1 + rate / 100.0, nper);
}

int main()
{

    mu::Parser p1;

    std::puts("\n====== EXPERIMENT 1 - Simple math expression =======");
    p1.SetExpr("2^1 + 2^3 + 2^4");
    std::cout << " [*] p1-A value: "
              << p1.GetExpr() << " = " << p1.Eval() << std::endl;

    p1.SetExpr("3.5 * 10 - sin(3.1415) * 2.0 + sqrt(10) / 100.0 + 2^3");
    std::cout << " [*] p1-B value: "
              << p1.GetExpr() << " = " << p1.Eval() << std::endl;

    std::puts("\n====== EXPERIMENT 2 - Expression with variables =======");
    p1.DefineConst("pi", 3.1415);
    double x = 10.0, y = 4.5;
    p1.DefineVar("x", &x);
    p1.DefineVar("y", &y);
    p1.SetExpr(" 3 * pi + sin(pi) + 4 * x + y - 5");
    std::cout << " [*] p1-C value: "
              << p1.GetExpr() << " = " << p1.Eval() << std::endl;

    std::puts("\n====== EXPERIMENT 3 - Expression with custom function =======");

    p1.DefineFun("fv", &future_value);

    p1.DefineFun("payoff", [](double S, double K){
        return std::max(S - K, 0.0);
    });

    p1.SetExpr("fv(100, 2, 8) + payoff(100, 90)");
    std::cout << " [*] p1-D value: " << p1.GetExpr() << " = " << p1.Eval() << std::endl;


    std::puts("\n====== EXPERIMENT 4 - Parser error handling =======");

    // When an error happens it throws an exception: mu::ParserError
    try{
        p1.SetExpr("10.2 * 5.0 + a * 3");
        double value = p1.Eval();
        std::cout << " [*] p1-E value: " << value << std::endl;
    } catch (mu::ParserError const& ex)
    {
        std::cerr << " [ERROR] p1-C Parser error: " << ex.GetMsg() << std::endl;
    }


    std::puts("\n====== EXPERIMENT 5 - Calutor Interactive Shell ======");

    if(isTTY_terminal())
    {
        std::cout << " === Calculator Started OK. =====" << std::endl;

        mu::Parser p2;
        double ans = 0.0;
        p2.DefineVar("ans", &ans);
        std::string line;

        while(std::cin.good())
        {
            std::cout << " EXPR => ";
            std::getline(std::cin, line);

            if(line == "")
                continue;

            if(line == "quit")
                break;

            p2.SetExpr(line);
            try {
                ans = p2.Eval();
                std::cout << " => ans = " << ans << "\n\n";
            } catch(mu::ParserError const& ex)
            {
                std::cerr << " [ERROR] Parser error " << ex.GetMsg() << std::endl;
            }
        }

    }

#if _WIN32
    std::cout << "Enter RETURN to exit. " << std::endl;
    std::cin.get();
#endif
    return 0;
}

Program output:

$ ./muparser1_formula 

====== EXPERIMENT 1 - Simple math expression =======
 [*] p1-A value: 2^1 + 2^3 + 2^4  = 26
 [*] p1-B value: 3.5 * 10 - sin(3.1415) * 2.0 + sqrt(10) / 100.0 + 2^3  = 43.0314

====== EXPERIMENT 2 - Expression with variables =======
 [*] p1-C value:  3 * pi + sin(pi) + 4 * x + y - 5  = 48.9246

====== EXPERIMENT 3 - Expression with custom function =======
 [*] p1-D value: fv(100, 2, 8) + payoff(100, 90)  = 127.166

====== EXPERIMENT 4 - Parser error handling =======
 [ERROR] p1-C Parser error: Unexpected token "a" found at position 13.

====== EXPERIMENT 5 - Calutor Interactive Shell ======
 === Calculator Started OK. =====
 EXPR => 9.81 * sin(3.1415 / 2.0) + 100 * sqrt(285.6) + exp(3.65)
 => ans = 1738.26

 EXPR => ans / 100.0 - 80.0
 => ans = -62.6174

 EXPR => ans * ans
 => ans = 3920.94

 EXPR => 

1.4 Exprtk - Math expression parser

Exprtk is a MIT-license single-header, header-only and turing-complete math expression parsing engine.

Official Web Site:

Official repository:

Features:

  • Header-only library in a single file.
  • Turing-complete
  • Sandboxed
  • Supports binding to defined-functions in C++-side.
  • Functions can operate on vectors
  • Control structures: for-loop; if-else; ternary operator.

Drawbacks:

  • The library exprtk.hpp has one megabyte (1 mb) in a single header file which significantly slows down the compile-time. Header-only design should only be used for small libraries.

Sample Project

  • GIST: https://gist.github.com/2ff870b653b2ec1519bf0423165db1c5
  • Note: The static library target 'mparser' makes the compile-time of the client code 'main.cpp' faster by using the PIMPL (Pointer-To-Implementation) design pattern and not exposing exprtk.hpp in the library header file.

File: CMakeLists.txt

cmake_minimum_required(VERSION 3.9)
project(exprk-parser)

#========== Global Configurations =============#
#----------------------------------------------#

set(CMAKE_CXX_STANDARD 17)     
set(CMAKE_VERBOSE_MAKEFILE ON)
set(CMAKE_CXX_EXTENSIONS OFF)

#----------- Add dependencies --------------------------#

#============= Functions and macros ===========================#
macro(Download_Single_Headerlib FILE URL)
    file(DOWNLOAD ${URL} ${CMAKE_BINARY_DIR}/include/${FILE})
    IF(NOT Download_Single_Headerlib_flag)
       include_directories(${CMAKE_BINARY_DIR}/include)
       set(Download_Single_Headerlib_flag TRUE)
    ENDIF()
endmacro()

Download_Single_Headerlib( exprtk.hpp 
                           https://github.com/ArashPartow/exprtk/raw/d81ac1a2ddd9877a7981d32c731fd9a75544ec68/exprtk.hpp )

#-----------  Target settings -------------------------------#

          add_library( mparser mparser.cpp mparser.hpp )
       add_executable( main main.cpp )
target_link_libraries( main mparser )

File: mparser.hpp (cmake target: mparser static library)

#ifndef _MPARSER_HPP_
#define _MPARSER_HPP_

#include <string>
#include <memory>
#include <functional>


// Unique-ownership smart pointer for implementing PimPl
// 
// Note: It was created because std::uniqe_ptr does 
//       not support 'incomplete types'.
//  
// Note: PimPl (Pointer-To-Implementation) idiom 
// is a technique for reducing compile-time and 
// maintaining ABI stability which mitigates the 
// fragile-ABI problem.
// 
template<typename T, void (*disposer) (T*)>
class pimpl_ptr
{
    T*       m_hnd;    
public:
    pimpl_ptr()
        : m_hnd(nullptr)
        //, m_disp(disp) 
        { }

    pimpl_ptr(T* hnd)
        : m_hnd(hnd)
        { }

    ~pimpl_ptr()
    {
        this->release();
    }

    // Disable copy constructor 
    pimpl_ptr(pimpl_ptr const&) = delete;
    // Disable copy-
    pimpl_ptr& operator=(pimpl_ptr const&) = delete;

    // Move Ctor 
    pimpl_ptr(pimpl_ptr&& rhs)
    {
        std::swap(m_hnd, rhs.m_hnd);
    }

    // Move assignment operator 
    pimpl_ptr& operator=(pimpl_ptr&& rhs)
    {
        std::swap(m_hnd, rhs.m_hnd);
    }

    T* get() const { return m_hnd; }

    void release()
    { 
        // Note: it is not possible to delete incomplete type 
        // in this way: 'delete m_hnd;'
        disposer(m_hnd);
        m_hnd = nullptr;
    }

    void reset(T* hnd)
    {
        this->release();
        m_hnd = hnd;
    }

    bool empty() const { return m_hnd == nullptr; }
    T&   operator* () const { return *m_hnd; }
    T*   operator-> () const { return m_hnd; }
};


class MathEvaluator
{
    struct impl;
    static void dispose(impl* p);
    pimpl_ptr<impl, MathEvaluator::dispose> m_pimpl;
public:
    MathEvaluator();
    ~MathEvaluator() = default;
    MathEvaluator& add_var(std::string name, double& ref);   

    // Register function pointer callback or non-capture lambda 
    MathEvaluator& add_function(std::string, double fptr (double) );
    // Register function of two variables 
    MathEvaluator& add_function(std::string, double fptr (double, double) );

    double  eval_code(std::string code);    
    bool    compile(std::string code);
    double  value();
    void    repl();
};

#endif

File: mparser.cpp (cmake target: mparser static library)

#include "mparser.hpp"

#include <iostream>
#include <string> 
#include <exprtk.hpp>

// --------------------------------------------------// 

struct MathEvaluator::impl 
{
    exprtk::expression<double>   expr;
    exprtk::symbol_table<double> symbol_table;
    exprtk::parser<double>       parser;     
    std::string                  code; 

    // Println function 
    exprtk::rtl::io::println<double> println{};

};

// static method 
void 
MathEvaluator::dispose(impl* p)
{
    delete p;
}

MathEvaluator::MathEvaluator(): m_pimpl(new impl)
{    
    m_pimpl->symbol_table.add_constants();
    m_pimpl->expr.register_symbol_table(m_pimpl->symbol_table);    
    m_pimpl->symbol_table.add_function("println", m_pimpl->println);
}

MathEvaluator& 
MathEvaluator::add_var(std::string name, double& ref)
{
    m_pimpl->symbol_table.add_variable(name, ref);
    return *this;
}

MathEvaluator& 
MathEvaluator::add_function(std::string name, double fptr (double) )
{
    m_pimpl->symbol_table.add_function(name, fptr);
    return *this;
}

MathEvaluator& 
MathEvaluator::add_function(std::string name, double fptr (double, double) )
{
    m_pimpl->symbol_table.add_function(name, fptr);
    return *this;
}

bool 
MathEvaluator::compile(std::string code)
 {
    bool r = m_pimpl->parser.compile(code, m_pimpl->expr);

    if(!r){ 
        std::string err = "Error: ";
        err = err + m_pimpl->parser.error();
        std::cerr << " Error: " << err << "\n";
         throw std::runtime_error(" [PARSER] Unable to parse expression.");
    }
    return r;
 }

double 
MathEvaluator::value()
{
    return m_pimpl->expr.value();
}

void MathEvaluator::repl()
{
    std::string line; 
    double result;
    while( std::cin.good() )
    {
        std::cout << " EXPRTK $ >> ";
        std::getline(std::cin, line);
        if(line.empty()) continue;
        if(line == "exit") return;
        try {
            this->compile(line);            
            std::cout << this->value() << '\n';
        } catch (std::runtime_error& ex) {
            std::cerr << "Error: " << ex.what() << '\n';

        }

    }
}

File: main.cpp (cmake target: main)

#include <iostream>
#include <string>
#include <cassert>

#include "mparser.hpp"

double myfun(double a, double b);
void   test_engine(MathEvaluator& engine, double& x);

int main(int argc, char** argv)
{
    std::puts(" [TRACE] I am up and running Ok. ");

    MathEvaluator engine;    
    double x = 1.0, y = 2.0, z = 3.0;
    engine.add_var("x", x)
        .add_var("y", y)
        .add_var("z", z)
        .add_function("myfun", &myfun);

    assert(argc == 2);

    auto command = std::string(argv[1]);
    if(command == "test" ){ test_engine(engine, x); }
    if(command == "repl" ){ engine.repl();          }

    std::cerr << " [TRACE] Shutdown engine Ok. " << '\n';
    return 0;
}

// -----------------------------------------//


double myfun(double a, double b)
{
    std::cerr << "  [TRACE] a = " << a << "\n";
    std::cerr << "  [TRACE] b = " << b << "\n";
    double r =  3.0 * a + 5.0 * b;
    std::cerr << "  [TRACE] result = " << r << "\n";
    std::cerr << "---------------------------------\n";
    return r;
}


void test_engine(MathEvaluator& engine, double& x)
{
    std::string code = R"( 
        // Define local variables 
        var a := 2.0 / exp(x) * x^2 + y;
        var b := 10.0 * sqrt(x) + z;

        // println('\n => x = ', x);
        // println('\n => y = ', y);

        // Call custom function
        var k := myfun(x, y);

        // Comment: the last expression is returned 
        4.0 * a + 3 * b + 10 * z + k;        
    )";        
    engine.compile(code);

    x = 3.0;
    std::cout << " => x = " << x << " ; engine = " << engine.value() << "\n";

    x = 5.0;
    std::cout << " => x = " << x << " ; engine = " << engine.value() << "\n";

    x = 15.0;
    std::cout << " => x = " << x << " ; engine = " << engine.value() << "\n";

    x = -15.0;
    std::cout << " => x = " << x << " ; engine = " << engine.value() << "\n";

    x = 20.0;
    std::cout << " => x = " << x << " ; engine = " << engine.value() << "\n";


    std::string code2 = R"( 
        // Vector/array variable 
        var xs [6] := {  2.0, 10.2,   -2.50,  9.256, 100.0,  25.0 };
        var ys [6] := { -2.0,  1.225, -5.56, 19.000, 125.0, 125.0 };

        println(' => xs =', xs);
        println(' => ys = ', ys);
        println(' => 3 * xs + 4 * ys = ', 3 * xs + 4 * ys);
        println(' => sum(xs) = ', sum(ys) );
        println(' => sum(ys) = ', sum(xs) );

    )";
    engine.compile(code2);
    engine.value();
}

Building

$ git clone https://gist.github.com/2ff870b653b2ec1519bf0423165db1c5 gist && cd gist 
$ cmake --config Debug -H. -B_build  
$ cmake --build _build --target 

Running

Running 'test' subcommand.

# Redirect std::cerr or stderr to file 
$ _build/main test 2> out.log
 [TRACE] I am up and running Ok. 
 => x = 3 ; engine = 121.546
 => x = 5 ; engine = 140.43
 => x = 15 ; engine = 218.19
 => x = -15 ; engine = -nan
 => x = 20 ; engine = 251.164
 => xs =   2.00000   10.20000   -2.50000    9.25600  100.00000   25.00000
 => ys =   -2.00000    1.22500   -5.56000   19.00000  125.00000  125.00000
 => 3 * xs + 4 * ys =   -2.00000   35.50000  -29.74000  103.76800  800.00000  575.00000
 => sum(xs) =  262.66500
 => sum(ys) =  143.95600

# View log file 
$ _build/main test 2> out.log
 [TRACE] I am up and running Ok. 
 => x = 3 ; engine = 121.546
 => x = 5 ; engine = 140.43
 => x = 15 ; engine = 218.19
 => x = -15 ; engine = -nan
 => x = 20 ; engine = 251.164
 => xs =   2.00000   10.20000   -2.50000    9.25600  100.00000   25.00000
 => ys =   -2.00000    1.22500   -5.56000   19.00000  125.00000  125.00000
 => 3 * xs + 4 * ys =   -2.00000   35.50000  -29.74000  103.76800  800.00000  575.00000
 => sum(xs) =  262.66500
 => sum(ys) =  143.95600

Running 'repl' subcommand for interactive shell.

 rlwrap _build/main repl 
 [TRACE] I am up and running Ok. 
 EXPRTK $ >> sin(pi) * 2.0  + 5.3 * cos(3/2 * pi)
-7.28665e-16
 EXPRTK $ >> exp(3.1)
22.198
 EXPRTK $ >> var a := exp(2.5); var b := a * 3.0 + 10.0; 3 * a + b
83.095
 EXPRTK $ >> 
 EXPRTK $ >> println(' x = ', x)
 x =    1.00000
0
 EXPRTK $ >> myfun(5.1, 2.56)
  [TRACE] a = 5.1
  [TRACE] b = 2.56
  [TRACE] result = 28.1
---------------------------------
28.1
 EXPRTK $ >> 
 EXPRTK $ >> myfun(5.1, 2.0 * pi + 10.0)
  [TRACE] a = 5.1
  [TRACE] b = 16.2832
  [TRACE] result = 96.7159
---------------------------------
96.7159
 EXPRTK $ >> 
 EXPRTK $ >> exit
 [TRACE] Shutdown engine Ok. 

1.5 Lua scripting engine

1.5.1 Overview

Lua (moon in Portuguese) is a ligthweight multi paradigm scripting language written in C. Due to Lua be available as small footprint libraryn this language is widely used as embedded scripting by many applications for scripting and as configuration or data description language.

Use cases:

  • Scripting for C or C++ applications
  • Extension language => Add new functionality and updates without recompilation.
  • Provide interactive REPL or shell to C or C++ applications.
  • Program configuration (settings)
  • Data description language

Some applications using Lua:

  • Nmap network scanner
  • MediaWiki (engine used by Wikipedia)
  • Nginx Web Server
  • Redis Database
  • Linux Conky
  • LuaTex
  • NetBSD
  • Cheat Engine
  • Geany text editor
  • Xmake building system for C, C++, Objective-C, Swift, Assembly, Golang, Rust, Dlang and Cuda
  • Lots of games use lua for scripting and allow non-programmers, such as end-users and game artists to contribute to the game development, create animations, movements, finite state machines and so on.
  • … …

More at:

Repository and C++ Binding Libraries

1.5.2 Further Reading

Documentation

Lua C API

Lua embedded in Geany Text Editor

Lua scripting in NMap Network Scanner

  • Script Language | Nmap Network Scanning
    • "The core of the Nmap Scripting Engine is an embeddable Lua interpreter. Lua is a lightweight language designed for extensibility. It offers a powerful and well-documented API for interfacing with other software such as Nmap. The second part of the Nmap Scripting Engine is the NSE Library, which connects Lua and Nmap. This layer handles issues such as initialization of the Lua interpreter, scheduling of parallel script execution, script retrieval and more. It is also the heart of the NSE network I/O framework and the exception handling mechanism. It also includes utility libraries to make scripts more powerful and convenient. The utility library modules and extensions are described in the section called 'NSE Libraries'."
  • Nmap Scripting Engine Source Code / GITHUB
  • Extending Nmap With Lua - DEV

Lua Scripting in Wireshark - Network Capture Application

Lua Scripting in XMake Building System

Lua scripting on NetBSD Kernel

Lua scripting on Redis Database

Lua scripting on Nginx Web Server

Lua scripting on Unreal Engine / game engine

Lua for DSL - Domain Specific Language

Videos

  • CppCon 2017: Andreas Weis "Howling at the Moon: Lua for C++ Programmers"
    • "C++ is a great tool for solving complex problems in a thorough way. But every once in a while, the desire for a simpler language emerges. For those parts of our code where performance is of secondary concern, but the ability to perform rapid iterations over the code is paramount, a scripting language might be a tempting choice. But integrating a second language besides C++ and managing the interaction between the two is also scary. Lua is a lightweight, dynamic language that was designed to be used as an embedded language within existing applications. It is easy to learn, has very reasonable runtime performance, and a memory footprint small enough that it is usable even on embedded systems. Furthermore, it is almost trivial to integrate with C++. This talk will give a brief introduction to the Lua scripting language, highlighting specifically how it can complement C++'s language features to enrich a developer's toolbox. In the second part of the talk, we will look at Lua's C API and give suggestions how to integrate it with a modern C++17 codebase. In particular we will focus on how to interface with the dynamic language Lua without compromising the benefits of C++'s strong type system."
  • CppCon 2018: JeanHeyd Meneide “Scripting at the Speed of Thought: Lua and C++ with sol3”
    • "A big part of accelerating development and promoting collaboration often translates to deferring a lot of the typical programmer work to a scripting language, to allow for those with more design-oriented ideas and experience to handle some of the workload. What happens, then, when you have to bind a scripting language like Lua into C++ to allow for this workflow? This session is going to be all about how you enable non-developers and developers alike to rapidly increase their development productivity by turning routines and algorithms into data that can be shipped alongside your product in Lua. We will talk primarily how you can use the library sol2 to abstract away the muck of working with the Lua C API and instead focus on using and describing both Lua and C++ with each other in a simple manner. We will demonstrate some of the more interesting properties of sol such as Overloading Support, Container Support, Usertypes – C++ classes made available with automatic support for unique/shared pointers – and Tables. By the time this session is over, you will have a firm grasp over what a good Lua Binding can offer you, and how you can accelerate your C++ development with it."

General

1.5.3 Example project with Sol2 C++ binding library

This sample project builds a C++ statically linked executable embedding the Lua scripting engine using the Sol2 binding library, which is header only. Neither Sol2 nor Lua libraries need to be installed before building this sample project as the CMake scripts take care of downloading and building all dependencies.

Lua repostiory mirror:

Sol2 library repository:

Sol2 library documentation:

Sample Project

GIST:

File: CMakeLists.txt

cmake_minimum_required(VERSION 3.0)
project(duktape-cc-trial)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_VERBOSE_MAKEFILE ON)

include(lua-lib.cmake)

#-----  Target Definitions ----------------------------#

       add_executable( embed-lua-sol embed-lua-sol.cpp)
target_link_libraries( embed-lua-sol lua::lualib )

# Lua REPL executable built from static library liblua.a (Linux)
# Note: the main() function is in the file main.c in the lua sources directory
       add_executable( lua-repl $<TARGET_OBJECTS:lua::lualib> )
target_link_libraries( lua-repl m pthread )

File: lua-lib.cmake

  • CMake Script for downloading sol2 binding library and lua library sources.
include(FetchContent)

# Note: the 'add_subriectory' line was commented becuyase 
#       library that will be downloaded does not have 
#       a CMakeListst.txt file at the root directory. 
macro(Download_Library_Git  NAME TAG REPOSITORY_URL)
    FetchContent_Declare(
        ${NAME}
        GIT_REPOSITORY  ${REPOSITORY_URL}
        GIT_TAG         ${TAG}
    )
    FetchContent_GetProperties(${NAME})
    if(NOT cpputest_POPULATED)
        FetchContent_Populate(${NAME})
        message("${NAME}_SOURCE_DIR} = ${${NAME}_SOURCE_DIR}")        

        # => Disable following line: the library does not have a CMakeLists.txt
        #    at the root directory.
        # add_subdirectory(${${NAME}_SOURCE_DIR} ${${NAME}_BINARY_DIR})
    endif()
endmacro()


# ====>> Download Lua library <<==========================#

Download_Library_Git( lua                       
                      v5.3.5
                      https://github.com/lua/lua
                    )

file(GLOB_RECURSE lua_sources "${lua_SOURCE_DIR}/*.c")
file(GLOB_RECURSE lua_headers" ${lua_SOURCE_DIR}/*.h")

message( [TRACE] " lua_SOURCE_DIR = ${lua_SOURCE_DIR} ")

               add_library( lua STATIC ${lua_sources} ${lua_headers} )
target_include_directories( lua PUBLIC ${lua_SOURCE_DIR} )

add_library( lua::lualib  ALIAS lua)

# ====>> Download Sol C++ binding library <<====================#

FetchContent_Declare( sol2 
                      GIT_REPOSITORY  https://github.com/ThePhD/sol2
                      GIT_TAG         v3.2.0
                    )

FetchContent_MakeAvailable( sol2 )
include_directories( ${sol2_SOURCE_DIR}/include )

File: embed-lua-sol.cpp

#include <iostream>
#include <string> 
#include <vector> 
#include <algorithm>

#include <sol/sol.hpp>


class Counter {
private: 
    std::string m_name;
    int         m_counter;

public: 

    // Ctor [1] => Default ctor 
    Counter(): Counter("untitled", 0) { }

    // Ctor [2]
    Counter(std::string name, int counter)
      : m_name{std::move(name)}, m_counter{counter}
    { 
        std::cout << " [TRACE] Counter created with =>  { " 
                  <<   " name = " << m_name 
                  << " ; counter = " << m_counter 
                  << " } \n";
    }

    int getCounter() const { return m_counter; }
    void setCounter(int n) {       
      m_counter = n; 
      std::cout << " [TRACE] I was set to value " << n << std::endl;
    }

    void increment() {       
      m_counter++; 
      std::cout << " [TRACE] increment event =>> counter = {  " 
                << m_name << " ; " << m_counter 
                << " } " << std::endl;
    }    
};

double add_fun(double a, double b)
{
    std::cout << " [TRACE] addfun() => a = " << a 
              << " ; b = " << b << std::endl;
    return a + b;             
}

void sol_eval(sol::state& ctx, std::string code)
{
    try {
        ctx.script( std::move(code) );        
    } catch ( sol::error const& ex) {
        std::cerr << " [SOL ERROR] " << ex.what() << "\n";
    }
}


int main()
{
    // Create an instance of lua Engine (aka virtual Machine)
    sol::state ctx{};

    // Load basic libraries (such as print)
    ctx.open_libraries(sol::lib::base, sol::lib::coroutine, sol::lib::string, sol::lib::io);    

    // Register function pointer 
    ctx.set_function("add_fun", &add_fun);

    // Register lambda object 
    ctx.set_function("make_vector", [](int n ) -> std::vector<int> {
        auto vec = std::vector<int>(n);
        for(int i = 0; i < n; i++ ) vec[i]= i * 3;
        return vec;
    });

    // Set variables in the Lua engine 
    ctx["points"]    = 2000;
    ctx.set("character", "<Marcus Tulius Cicero>");

    /* ===========>>> E X P E  R I M E N T / 1  <<=========*/
    std::puts("\n [EXPERIMENT 1] ==>> Evaluating string as code ");
    {
        // ===>>> Eval code as string <<=== 
        // Throws exception sol::error 
        ctx.script(R"(
            print(" [LUA] Hello world lua "); 

            x = add_fun(10, 20);
            print("\n  [LUA] result =  " .. x);

            v = make_vector(5);
            print("\n  [LUA] Printing a vector ");

            for i = 1, 5 do 
                print("   -> v[" .. i .. " ] = " .. v[i] );
            end 

            print("  [LUA] VAR points    = " .. points );
            print("  [LUA] VAR character = " .. character );
        )");

    }

    /* ===========>>> E X P E  R I M E N T / 2  <<=========*/
    std::puts("\n\n [EXPERIMENT 2] ==>> Reading configuration ");
    {
        ctx.script(R"(
            -- Simulation of user configuration from script     
            asset_path   = "C:\\\\Users\\myuser\\data\\files\\";
            user_credits = 2000;
            width        = 200.561;        
        )");

        auto asset_path = ctx.get<std::string>("asset_path");
        int user_credits = ctx["user_credits"];

        std::cout << "  [*] => asset_path = " << asset_path << "\n";
        std::cout << "  [*] => user_credits = " << user_credits << "\n";

    }

    /* ===========>>> E X P E  R I M E N T / 3  <<=========*/
    std::puts("\n\n [EXPERIMENT 3] ==>> Register C++ classes ");

    struct StatefulFunctor {
        int state = 0;
        StatefulFunctor(int state): state(state){ }
        int operator()(){ 

            std::cout << "  *=>> [StatefulFunctor] My new state is = " 
                      << this->state << "\n";
            return state++; 
        }
    };

    auto stateful = StatefulFunctor(10);
    ctx.set_function("stateful", stateful);

    ctx.script(R"(
        stateful();
        stateful();
        stateful();
    )");


    ctx.new_usertype<Counter>(
        // Class name 
         "Counter"          

        //  --- Register methods  ------ //
        ,"getCounter", &Counter::getCounter
        ,"setCounter", &Counter::setCounter
        ,"increment",  &Counter::increment

        // --- Register properties  ---- //
        , "value",     sol::property( &Counter::getCounter
                                    , &Counter::setCounter)
    );


    sol_eval(ctx, R"(
        print("\n ----->>> Calling C++ classes from Lua <----- ");

        -- Create new instance (object) of C++ class Counter 
        counter = Counter.new();
        counter:increment();
        counter:increment(); 
        counter:increment();

        x = counter:getCounter(); 
        print("  [*] value of counter is equal to = " .. x);

        counter.value = 2000;
        print(" [*] Counter value is equal to = " .. counter.value );
    )");

    Counter* ptr = ctx.get<Counter*>("counter");

    std::cout << " [FROM C++] counter value = " 
              << ptr->getCounter() 
              << "\n";

    return 0;
}

Building => Debug build:

$ git clone https://gist.github.com/17a37d905d3d71c0ae66661a189481b5 lua-sol && cd lua-sol 
$ cmake --config Debug -H. -B_build 
$ cmake --build _build --target 

Building => Release build:

$ git clone https://gist.github.com/17a37d905d3d71c0ae66661a189481b5 lua-sol && cd lua-sol 
$ cmake --config Release -H. -B_build 
$ cmake --build _build --target 

Check executable:

# Confirm whether the executable is statically linked against LuaLib 
$ ldd _build/embed-lua-sol 
       linux-vdso.so.1 (0x00007ffdf0fc4000)
       libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fe4490e6000)
       libm.so.6 => /lib64/libm.so.6 (0x00007fe448fa0000)
       libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fe448f85000)
       libc.so.6 => /lib64/libc.so.6 (0x00007fe448dbb000)
       /lib64/ld-linux-x86-64.so.2 (0x00007fe4492fa000)

  # Static library 
  $ file _build/liblua.a 
 _build/liblua.a: current ar archive

Program output:

$ _build/embed-lua-sol 

[EXPERIMENT 1] ==>> Evaluating string as code 
[LUA] Hello world lua 
[TRACE] addfun() => a = 10 ; b = 20

 [LUA] result =  30.0

 [LUA] Printing a vector 
  -> v[1 ] = 0
  -> v[2 ] = 3
  -> v[3 ] = 6
  -> v[4 ] = 9
  -> v[5 ] = 12
 [LUA] VAR points    = 2000
 [LUA] VAR character = <Marcus Tulius Cicero>


[EXPERIMENT 2] ==>> Reading configuration 
 [*] => asset_path = C:\\Users\myuser\data\files\
 [*] => user_credits = 2000


[EXPERIMENT 3] ==>> Register C++ classes 
 *=>> [StatefulFunctor] My new state is = 10
 *=>> [StatefulFunctor] My new state is = 11
 *=>> [StatefulFunctor] My new state is = 12

----->>> Calling C++ classes from Lua <----- 
[TRACE] Counter created with =>  {  name = untitled ; counter = 0 } 
[TRACE] increment event =>> counter = {  untitled ; 1 } 
[TRACE] increment event =>> counter = {  untitled ; 2 } 
[TRACE] increment event =>> counter = {  untitled ; 3 } 
 [*] value of counter is equal to = 3
[TRACE] I was set to value 2000
[*] Counter value is equal to = 2000
[FROM C++] counter value = 2000

Run Lua repl executable (defined in CMake):

$ rlwrap  _build/lua-repl 
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
> 
> 

> print(" Hello world Lua / Luna / Moon REPL ")
 Hello world Lua / Luna / Moon REPL 

> for i = 1, 5 do print(" i = " .. i ) end
 i = 1
 i = 2
 i = 3
 i = 4
 i = 5


function myfunction(a, b) 
  return math.sin(a) * math.exp(b) / a - a * b 
end 

> myfunction(3.5, 2.0)
-7.7405591279893

function add (a)
   local sum = 0
   for i,v in ipairs(a) do
      sum = sum + v
   end
   return sum
end

> add({ 2.5, 10.2, -2.51, 8.251, 10.56})
29.001

function add (a)
      local sum = 0
      for i,v in ipairs(a) do
        sum = sum + v
      end
return su

1.6 Squirrel Scripting Language

1.6.1 Overview

  • Squirrel is a embedded scripting language, similar to Lua, but with C-like syntax, designed to be embedded in larger C or C++ applications such as game engines. Squirrel is written in C++, but it only exposes a C API, which makes binding C++ code cumbersome. However, there are many libraries which simplifies the embedding of squirrel in C++ codebases.

Official Web Site

Official Repository

Squirrel fork with a more C++-like syntax

Articles about squirrel language

Applications using Squirrel

Libraries for simplifying embedding squirrel in C++ code

Libraries for simplifying squirrel embedding in C++ code (binding C++ code):

1.6.2 Building Squirrel standalone REPL interpreter

Download and build:

$ mkdir ~/build && cd build 
$ git clone https://github.com/albertodemichelis/squirrel
$ cmake -H. -B_build -DCMAKE_BUILD_TYPE=Debug 
$ cmake --build _build --target

Play with squirrel interactive shell (REPL):

$ _build/bin/sq
Squirrel 3.1 stable Copyright (C) 2003-2017 Alberto Demichelis (64 bits)

sq> print(" === Hello world Squirrel === ")
 === Hello world Squirrel === 

sq> function add_to_10(x){ return x + 10; }

sq>print(add_to_10(25))
35

sq> x <- cos(3.1415 / 2) + 10 

sq>print(" x = " + x.tostring())
 x = 10

sq> for(local i = 0; i < 5; i++) print(" \n [TRACE] i = " + i.tostring());

 [TRACE] i = 0 
 [TRACE] i = 1 
 [TRACE] i = 2 
 [TRACE] i = 3 
 [TRACE] i = 4

1.6.3 Example - embedding Squirrel with Squall Library

This example demonstrates how to embed the Squirrel programming language in a C++ application using the Squall header-only library.

  • Squal Repository: https://github.com/jonigata/squall
  • Note: This project is self-contained, no library needs to be installed on the system as Squall automatically fetches Squirrel sources using Cmake FetchContent.

Sample Project

File: CMakeLists.txt

cmake_minimum_required (VERSION 3.11)
project(squirrel_squall_test)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED on)
set(BUILD_EXAMPLES off)

# -----------------------------------------------#
include(FetchContent)

FetchContent_Declare(
  squall 
  URL      https://github.com/jonigata/squall/archive/master.zip
  )

FetchContent_MakeAvailable(squall)

#-------- TARGET DEFINITIONS --------------------#
message([TRACE] " squall_SOURCE_DIR = ${squall_SOURCE_DIR}/squall  ")

            add_executable ( squirrel-test squirrel_test.cpp)
     target_link_libraries ( squirrel-test squirrel_static sqstdlib_static)
target_include_directories ( squirrel-test PUBLIC
                               ${squirrel_SOURCE_DIR}/include
                               ${squall_SOURCE_DIR}
                               )

File: squirrel_test.cpp

#include <iostream>
#include <string>
#include <algorithm>
#include <vector> 

#include <squall/squall_vmstd.hpp>
#include <squall/squall_klass.hpp>

void some_cpp_fun(int n )
{
    for(int i = 0; i < n; i++)
        std::printf("\n   [some_cpp_function] => i = %d ", i);
}

class ChartXY
{
    int m_width; 
    int m_height;
public:
    ChartXY(): m_width(20), m_height(50) 
    {
        std::cout << " [ChartXY] Ctor() - I was created!. OK. " << std::endl;
    }

    void set_width(int x){ m_width = x; }
    void set_height(int x){ m_height = x; }

    void draw() const 
    { 
        std::printf(" [ChartXY] draw() => Draw chart with: width = %d ; height = %d"
                    , m_width, m_height);
    }
};

int main()
{
    // Create a virtual-machine object for Squirrel language 
    // Note: throws squall::squirrel_error
    squall::VMStd vm; 

    // ------------------------------------------------------------------------//
    // [EXPERIMENT 1] Evaluate scripts provided as strings                     //
    //-------------------------------------------------------------------------//
    std::puts(" =>> [EXPERIMENT] 1 - Evaluating code as string. ");  
    std::puts(" ---------------------------------------------\n");

    try {

        vm.dostring(R"( 
            // --- Squirrel Comment ----- // 
            print(" <SQUIRREL>  =>> Hello world squirrel!");

            function myfunc(x) {  
                local a = x + 5;
                local b = 7 * a + x;
                return b - a + 10;  
            }

            function myfunc2() {
                print(" \n  <SQUIRREL> I was called by the C++ code ");
            }

            print("\n <SQUIRREL> =>> myfunc(4) = " + myfunc(4).tostring() );

            print("\n\n <SQUIRREL> --- For Loop test ---- ");
            for(local i = 0; i < 5; i++) { 
                 print("\n   i = " + i.tostring() );  
            }
        )");

    } catch( squall::squirrel_error const& ex )
    {
        std::cerr << "\n [SQUIRREL ERROR] Error =>  " << ex.what() << std::endl;        
    }

    // ------------------------------------------------------------------------//
    // [EXPERIMENT 2] Evaluate scripts provided as strings                     //
    //-------------------------------------------------------------------------//
    std::puts("\n =>> [EXPERIMENT] 2 - Getting variables defined in the code.");  
    std::puts(" -----------------------------------------------------------\n");

    {
        vm.dostring(R"( 
            // ---- Global varibles for configuration ------ // 
            ::myvar_width <- 100;
            ::myvar_float <- 122.56161;
            ::myvar_string <- "/path/to/interpreter.exe"; 
        )");

        squall::TableBase table = vm.root_table();
        auto myvar_float = table.get<float>("myvar_float");       
        auto myvar_width = table.get<int>("myvar_width");
        auto myvar_string = table.get<std::string>("myvar_string");
        std::cout << "  =>>  myvar_width = " << myvar_width << std::endl;
        std::cout << "  =>>  myvar_float = " << myvar_float << std::endl;
        std::cout << "  =>> myvar_string = " << myvar_string << std::endl;
    } 

    // ------------------------------------------------------------------------//
    // [EXPERIMENT 3] Call functions defined in the script (Virtual Machine )  //
    //-------------------------------------------------------------------------//    
    std::puts("\n\n =>> [EXPERIMENT] 3 - Calling functions defined in the script (VM)");
    std::puts(" ------------------------------------------------------------------\n");    
    // Throws: 'squall::squirrel_error' 
    int result = vm.call<int>("myfunc", 10);
    std::cout << "   =>>> myfunc(4) = " << result << std::endl;

    vm.call<void>("myfunc2");

    // ------------------------------------------------------------------------//
    // [EXPERIMENT 4] Call C++ functions from the script                       //
    //-------------------------------------------------------------------------//    
    std::puts("\n\n =>> [EXPERIMENT] 4 - Calling functions defined in the script (VM)");
    std::puts(" ------------------------------------------------------------------\n");

    // Register C++ function pointer 
    vm.defun("some_cpp_fun", &some_cpp_fun);

    vm.dostring(R"(
        print(" \n [SQUIRREL] => Call C++ function some_cpp_fun() ");
        some_cpp_fun(5);
     )");


    // Register C++ lambda object 
    vm.defun("call_me", [=](std::string const& param) {
        std::cout << "\n [TRACE] call_me() Parameter = " << param << "\n";
        return  " name = " + param;
    });

    vm.dostring(R"(
        local x = call_me("<SQUIRREL-INTERPRETER>");
        print(" [SQUIRREL] \n x <- " + x);
     )");

    // ------------------------------------------------------------------------//
    // [EXPERIMENT 5] Call C++ classes from Squirrel-side                     //
    //-------------------------------------------------------------------------//    
    std::puts("\n\n =>> [EXPERIMENT] 5 - Calling C++ classes from Squirrel        ");
    std::puts(" ------------------------------------------------------------------\n");

    // Create metaobject 'k' that describes ChartXY class 
    squall::Klass<ChartXY> k(vm, "ChartXY");
    k.func("set_width",  &ChartXY::set_width);
    k.func("set_height", &ChartXY::set_height);
    k.func("draw",       &ChartXY::draw);

    vm.dostring(R"( 
        function manipulate_chart(ch){           
            ch.set_width(25);
            ch.set_height(10);
            ch.draw();
        }

        function draw_with(ch, w, h)
        {
            print(" \n [SQUIRREL LOG] Function draw_with called. OK. \n");
            ch.set_width(w);
            ch.set_height(h);
            ch.draw();
        }
     )");

    ChartXY mychart;
    vm.call<void>("manipulate_chart", &mychart);
    vm.call<void>("draw_with", &mychart, 100, 200);

    std::cout << "\n\n";

    squall::TableBase table = vm.root_table();

    // Pass object to Squirrel side 
    table.set("mychart", mychart);

    vm.dostring(R"(
        mychart.set_width(250);
        mychart.set_width(600);
        mychart.draw();
    )");


#if 0  
    // Segmentation Falt Coredump if the C++ object 
    // is created on the Squirrel-side.
    vm.dostring(R"(
        local c = ChartXY();
        c.set_width(150);
        c.set_height(175);
        c.draw();
    )");
#endif 

    return 0;
}

Build:

$ cmake -H. -B_build -DCMAKE_BUILD_TYPE=Debug 
$ cmake --build _build --target 

Check executable dependencies:

$ ldd _build/squirrel-test 
       linux-vdso.so.1 (0x00007ffe34dd6000)
       libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fd3ebd2a000)
       libm.so.6 => /lib64/libm.so.6 (0x00007fd3ebbe4000)
       libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fd3ebbc9000)
       libc.so.6 => /lib64/libc.so.6 (0x00007fd3eb9ff000)
       /lib64/ld-linux-x86-64.so.2 (0x00007fd3ebf3e000)

Run application:

$ _build/squirrel-test 

=>> [EXPERIMENT] 1 - Evaluating code as string. 
---------------------------------------------

<SQUIRREL>  =>> Hello world squirrel!
<SQUIRREL> =>> myfunc(4) = 68

<SQUIRREL> --- For Loop test ---- 
  i = 0
  i = 1
  i = 2
  i = 3
  i = 4
=>> [EXPERIMENT] 2 - Getting variables defined in the code.
-----------------------------------------------------------

 =>>  myvar_width = 100
 =>>  myvar_float = 122.562
 =>> myvar_string = /path/to/interpreter.exe


=>> [EXPERIMENT] 3 - Calling functions defined in the script (VM)
------------------------------------------------------------------

  =>>> myfunc(4) = 110

 <SQUIRREL> I was called by the C++ code 

=>> [EXPERIMENT] 4 - Calling functions defined in the script (VM)
------------------------------------------------------------------


[SQUIRREL] => Call C++ function some_cpp_fun() 
  [some_cpp_function] => i = 0 
  [some_cpp_function] => i = 1 
  [some_cpp_function] => i = 2 
  [some_cpp_function] => i = 3 
  [some_cpp_function] => i = 4 
[TRACE] call_me() Parameter = <SQUIRREL-INTERPRETER>
[SQUIRREL] 
x <-  name = <SQUIRREL-INTERPRETER>

=>> [EXPERIMENT] 5 - Calling C++ classes from Squirrel        
------------------------------------------------------------------

[ChartXY] Ctor() - I was created!. OK. 
[ChartXY] draw() => Draw chart with: width = 25 ; height = 10 
[SQUIRREL LOG] Function draw_with called. OK. 
[ChartXY] draw() => Draw chart with: width = 100 ; height = 200

[ChartXY] draw() => Draw chart with: width = 600 ; height = 200


1.7 Duktape - Embeddable Javascript Engine

1.7.1 Overview

  • Duktape is a small footprint embeddable Javascript (ECMAScript) engine, written in C, which can be used for providing scripting capabilities for C or C++ applications.
  • License: MIT
  • Possible Use Cases:
    • Configuration
    • Data description language
    • User plugins
    • User extensions
    • Scripting for games
  • Features:
    • Embeddable, portable, compact: can run on platforms with 160kB flash and 64kB RAM
    • Built-in debugger
    • Built-in regular expression engine
    • Minimal, retargetable platform dependencies
    • Combined reference counting and mark-and-sweep garbage collection with finalization
    • Bytecode dump/load for caching compiled functions
    • Distributable includes an optional logging framework, CommonJS-based module loading implementations, etc

Official Website

Official Repository

C++ Binding Libraries

1.7.2 Example project with DukGlue C++ binding library

This following project CMakeLists.txt automatically downloads dukglue binding library and duktape engine sources and builds a C++ demonstration code embedding duktape JavaScript engine.

  • DukGlue Binding Library: https://github.com/Aloshi/dukglue
    • Advantage:
      • Easy to use and lots of examples.
    • Drawbacks:
      • Lack of namespaces which enhances API discoverability
      • Lack of C++ wrappers to some Duktape C-types
      • Lack of a CMakeLists.txt at the top directory.

File: CMakeLists.txt

cmake_minimum_required(VERSION 3.0)
project(duktap-embed)

include(FetchContent)

# Download library archive (zip, *.tar.gz, ...) from URL
macro(Download_Library_Url NAME URL)
  FetchContent_Declare(${NAME} URL  ${URL})
  FetchContent_GetProperties(${NAME})
  if(NOT ${NAME}_POPULATED)
    FetchContent_Populate(${NAME})
   # add_subdirectory(${${NAME}_SOURCE_DIR} ${${NAME}_BINARY_DIR})
  endif()
endmacro()


# ====>> Duktape JavaScript Engine Configuration <<===========#

Download_Library_Url(duktape
  "https://duktape.org/duktape-2.5.0.tar.xz"
  )

# FetchContent_MakeAvailable(duktape)

message( [TRACE] " =>> duktape_SOURCE_DIR = ${duktape_SOURCE_DIR} ")


file(GLOB_RECURSE duktape_sources "${duktape_SOURCE_DIR}/src/*.c")
file(GLOB_RECURSE duktape_headers "${duktape_SOURCE_DIR}/src/*.h")

message( [TRACE] " duktape_sources = ${duktape_sources} ")

              add_library (duktape ${duktape_sources} ${duktape_headers} )
target_include_directories(duktape PUBLIC ${duktape_SOURCE_DIR}/src  )

# ----------- DukGlue Library ----------------------------#

FetchContent_Declare(
  dukglue 
  URL       https://github.com/Aloshi/dukglue/archive/master.zip
  )

FetchContent_MakeAvailable(dukglue)

#----- Main Target Definition ----------------------------#
add_executable(duktape-embed duktape-embed.cpp)
target_link_libraries(duktape-embed duktape dukglue)

File: duktape-embed.cpp

#include <iostream>
#include <string>
#include <vector>
#include <cassert> 

// Repository: https://github.com/Aloshi/dukglue
#include <dukglue/dukglue.h>

void print_number(int x)
{
  std::cout << " [TRACE] number passed is = " << x << std::endl;
}

void log_text(std::string const& text)
{
    std::cout <<  " =>> [C++-LOG] - " << text << std::endl;
}

int eval_code(duk_context* ctx, std::string const& code)
{
    return duk_peval_string(ctx, code.c_str());
}

void plot_points(std::vector<float> const& points)
{
  std::cout << "  =>> [TRACE] Plot points  =>> ";
  for(auto const& x: points) { std::cout << " x = " << x; }
  std::cout << " \n";
}

class Counter {
private: 
    std::string m_name;
    int         m_counter;

public: 

    // Ctor [1] => Default ctor 
    Counter(): Counter("untitled", 0) { }

    // Ctor [2]
    Counter(std::string name, int counter)
      : m_name{std::move(name)}, m_counter{counter}
    { 
        std::cout << " [TRACE] Counter created with =>  { " 
                  <<   " name = " << m_name 
                  << " ; counter = " << m_counter 
                  << " } \n";
    }

    int getCounter() const { return m_counter; }
    void setCounter(int n) {       
      m_counter = n; 
      std::cout << " [TRACE] I was set to value " << n << std::endl;
    }

    void increment() {       
      m_counter++; 
      std::cout << " [TRACE] increment event =>> counter = {  " 
                << m_name << " ; " << m_counter 
                << " } " << std::endl;
    }    

};


int main()
{
      // Create Duktape Virtual machine 
      duk_context* ctx = duk_create_heap_default();

      /* ========================== EXPERIMENT 1 =============*/
      std::puts("\n === [EXPERIMENT 1] ==>> Register and call C++ functions <<===== ");
      {
          // Register pointer to functions function (function pointer) in 
          // the JS engine (aka virtual machine)
          dukglue_register_function(ctx, &print_number, "print_number"); 
          dukglue_register_function(ctx, log_text, "log_text");
          dukglue_register_function(ctx, plot_points, "plot_points");

          const char* code1 = R"(
              print_number(10);
              log_text(" Hello world from Javascript" ); 
              log_text(" Toke is equal to " + 100 ); 
              log_text( " " + 1000 );      

              plot_points( [ 20.5, 100.23, -125.254, 8.251, 100.0 ]);
          )";

          // Evaluate code, returns false on error 
          auto n = eval_code(ctx, code1);

          if(n) { std::cerr << " [ERROR] A duktape evaluation error has happened. "  << std::endl; }

      }

      /* ========================== EXPERIMENT 2 ====================*/
      std::puts("\n === [EXPERIMENT 2] ==>> Register and call C++ classes <<===== \n");
      {
          // Register class counter 
          dukglue_register_constructor<Counter>(ctx, "Counter");      
          dukglue_register_constructor<Counter, std::string, int>(ctx,  "Counter");     
          dukglue_register_method(ctx, &Counter::getCounter , "getCounter");
          dukglue_register_method(ctx, &Counter::setCounter , "setCounter");
          dukglue_register_method(ctx, &Counter::increment , "increment");

          dukglue_register_property(ctx                   // Pointer to engine (VM)
                                  , &Counter::getCounter  // Getter 
                                  , &Counter::setCounter  // Setter 
                                  , "number"              // Property name 
                                  );

          int ret = eval_code(ctx, R"( 
              var counter = new Counter("mycounter", 10); 

              for(i = 0 ; i < 5; i++) { counter.increment(); }

              var n = counter.getCounter(); 
              log_text(" [BEFORE] Counter value = " + n );

              counter.setCounter(100);
              log_text(" [AFTER 1 ] Counter value = " + counter.getCounter() );

              counter.number = 400;
              log_text(" [AFTER 2] Counter value = " + counter.number );
          )");
          assert( ret == 0 );

      }

      /* ======= Calling Javascript Engine from C++ ====================*/
      // Note: It is useful for reading data or user configuration 
      std::puts("\n === [EXPERIMENT 3] ==>> Calling engine objects from C++ <<===== \n");
      {
          const char* code = R"(
            // Global variables for configuration 
            points = 200; 
            asset_path = "C:\\\\Users\\dummy\\data\\graphics";

            function my_js_function(n){
                log_text( " <my_js_function> =>> n = " + n );
                var k = 20 * n + 100;
                return k; 
            }

          )";
          eval_code(ctx, code);

          // Throws error: DukErrorException
          auto points = dukglue_peval<int>(ctx, "points");
          std::cout << "  [*] =>> points = " << points << std::endl;

          // Throws error: DukErrorException
          auto asset_path = dukglue_peval<std::string>(ctx, "asset_path");
          std::cout << "  [*] =>> asset_path = " << asset_path << std::endl;

          auto jsexpr = dukglue_peval<double>(ctx, "3.51 * 10.52 - 8.251 / 100");
          std::cout << "  [*] jsexpr = " << jsexpr << std::endl;

          // Call Javascript function from C++ 
          auto func = dukglue_peval<DukValue>(ctx, "my_js_function");
          int res = dukglue_pcall<int>(ctx, func, 20);
          std::cout << "  [*] res = " << res << std::endl;

      }

    // Release Javascript engine object (aka virtual machine)
    ::duk_destroy_heap(ctx);

    return 0;
}

Build:

$ cmake -H. -B_build -DCMAKE_BUILD_TYPE=Debug
$ cmake --build _build --target 

Program output:

$ ./build/duktape-embed 

=== [EXPERIMENT 1] ==>> Register and call C++ functions <<===== 
[TRACE] number passed is = 10
=>> [C++-LOG] -  Hello world from Javascript
=>> [C++-LOG] -  Toke is equal to 100
=>> [C++-LOG] -  1000
 =>> [TRACE] Plot points  =>>  x = 20.5 x = 100.23 x = -125.254 x = 8.251 x = 100 

=== [EXPERIMENT 2] ==>> Register and call C++ classes <<===== 

[TRACE] Counter created with =>  {  name = mycounter ; counter = 10 } 
[TRACE] increment event =>> counter = {  mycounter ; 11 } 
[TRACE] increment event =>> counter = {  mycounter ; 12 } 
[TRACE] increment event =>> counter = {  mycounter ; 13 } 
[TRACE] increment event =>> counter = {  mycounter ; 14 } 
[TRACE] increment event =>> counter = {  mycounter ; 15 } 
=>> [C++-LOG] -  [BEFORE] Counter value = 15
[TRACE] I was set to value 100
=>> [C++-LOG] -  [AFTER 1 ] Counter value = 100
[TRACE] I was set to value 400
=>> [C++-LOG] -  [AFTER 2] Counter value = 400

=== [EXPERIMENT 3] ==>> Calling engine objects from C++ <<===== 

 [*] =>> points = 200
 [*] =>> asset_path = C:\\Users\dummy\data\graphics
 [*] jsexpr = 36.8427
=>> [C++-LOG] -  <my_js_function> =>> n = 20
 [*] res = 500


1.7.3 Example project with Duktape-CC binding library

  • Duktape-CC binding library: https://github.com/stfwi/duktape-cc/
    • Benefits
      • Namespace
      • RAII for duktape C-API
      • Javascript common known APIs such as console.log()
    • Disadvantage:
      • No possible to bind lambda function.
      • No possible to bind C++ classes or objects
      • No CMakeLists.txt at top directory, which makes the library usage easier, but the following cmake scripts solves this problem.

File: CMakeLists.txt

cmake_minimum_required(VERSION 3.0)
project(duktape-cc-trial)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_VERBOSE_MAKEFILE ON)

include(duktape.cmake)

#----- Main Target Definition ----------------------------#
       add_executable( duktape-script duktape-script.cpp)
target_link_libraries( duktape-script duktape-cc )

File: duktape.cmake

include(FetchContent)

# Note: the 'add_subriectory' line was commented becuyase 
#       library that will be downloaded does not have 
#       a CMakeListst.txt file at the root directory. 
macro(Download_Library_Git  NAME TAG REPOSITORY_URL)
    FetchContent_Declare(
        ${NAME}
        GIT_REPOSITORY  ${REPOSITORY_URL}
        GIT_TAG         ${TAG}
    )
    FetchContent_GetProperties(${NAME})
    if(NOT cpputest_POPULATED)
        FetchContent_Populate(${NAME})
        message("${NAME}_SOURCE_DIR} = ${${NAME}_SOURCE_DIR}")        

        # => Disable following line: the library does not have a CMakeLists.txt
        #    at the root directory.
        # add_subdirectory(${${NAME}_SOURCE_DIR} ${${NAME}_BINARY_DIR})
    endif()
endmacro()


# ====>> Duktape JavaScript Engine Configuration <<===========#

Download_Library_Git( duktape-cc 
                      51fed200b0c3353a60fa560aa8a13a480f0ec0c7
                      https://github.com/stfwi/duktape-cc/
                    )

file(GLOB_RECURSE duktape_sources "${duktape-cc_SOURCE_DIR}/duktape/*.c")
file(GLOB_RECURSE duktape_headers1 "${duktape-cc_SOURCE_DIR}/duktape/*.hh")
file(GLOB_RECURSE duktape_headers2 "${duktape-cc_SOURCE_DIR}/duktape/*.h")

               add_library( duktape-cc ${duktape_sources} ${duktape_headers1} ${duktape_headers2} )
target_include_directories( duktape-cc PUBLIC ${duktape-cc_SOURCE_DIR} )

File: duktape-script.cpp

#include <iostream> 
#include <string> 
#include <vector> 
#include <fstream>

#include <duktape/duktape.hh>
#include <duktape/mod/mod.stdio.hh>

int main()
{
    std::puts(" [TRACE] Program started Ok. ");

    // Create Duktape Engine object (Virtual Machine)
    auto ctx = duktape::engine{};

    // Load all functions from stdio module 
    // ==> Note: It is necessary for console.log() work 
    duktape::mod::stdio::define_in(ctx);   


    std::puts("\n [EXPERIMENT 1] ======= Evaluate string as code ========");

    ctx.eval<void>(R"( 
        console.log(" [INFO] Hello world Javascript Engine ");

        var i = 0;
        while(i < 5) {
            console.log(" [TRACE] <ducktape>  i = " + i);
            i++;
        }
    )");

    std::puts("\n [EXPERIMENT 2] == Read/write values to the engine the engine =");

    // Write or pass values to the engine. 
    ctx.define("app.version", "0.251");
    ctx.define("user.points", 1000);
    ctx.define("array1", std::vector<double>{ 4.51, 9.25, -25.154, 205.2 });
    ctx.define("array2", std::vector<std::string>{ "C++", "ADA-Spark", "Rust", "Dlang", "OCaml" });

    std::string script_file = "/tmp/myscript.js";

    const char* script_code = R"(
        console.log("  => app.version = " + app.version );
        console.log("  => user.points = " + user.points );
        console.log("  => array1 = " + array1);
        console.log("  => array2 = " + array2);

        myconfig_path = "/Users/data/osx/config";
        user_credits = 1020; 
        vector = [100.25, 90.251, -120.5150];
    )";

    // Write script code to file     
    auto fs = std::ofstream(script_file);
    // Flush forces writing to the IO
    fs << script_code << std::flush;          
    // Execute script from file 
    ctx.include(script_file);

    std::cout << " ---- Read configuration from file " << std::endl; 

    // Throws exception: duktape::detail::basic_script_error<void>
    auto myconfig_path = ctx.eval<std::string>("myconfig_path");
    auto credits       = ctx.eval<int>("user_credits");
    auto vec           = ctx.eval<std::vector<double>>("vector");
    std::cout << "\n\n[*] my_config_path = " << myconfig_path << "\n";
    std::cout << "[*]   user_credits = " << credits << "\n";
    std::cout << "[*] vec[0] = " << vec[0] << " ; vec[1] = " << vec[1] << "\n";    

    return 0;
}

Build:

$ cmake -H. -B_build -DCMAKE_BUILD_TYPE=Debug
$ cmake --build _build --target 

Program output:

 $ _build/duktape-script 
 [TRACE] Program started Ok. 

 [EXPERIMENT 1] ======= Evaluate string as code ========

 [EXPERIMENT 2] == Read/write values to the engine the engine =
 [INFO] Hello world Javascript Engine 
 [TRACE] <ducktape>  i = 0
 [TRACE] <ducktape>  i = 1
 [TRACE] <ducktape>  i = 2
 [TRACE] <ducktape>  i = 3
 [TRACE] <ducktape>  i = 4
  => app.version = 0.251
  => user.points = 1000
  => array1 = 4.51,9.25,-25.154,205.2
  => array2 = C++,ADA-Spark,Rust,Dlang,OCaml
 ---- Read configuration from file 


[*] my_config_path = /Users/data/osx/config
[*]   user_credits = 1020
[*] vec[0] = 100.25 ; vec[1] = 90.251

1.8 QuickJS - ES20 Javascript Engine

QuickJS is small and lightweight embeddable Javascript engine, written in C, which supports ES2020 technical specification. This engine was created by Fabrice Bellard, creator of many widely used open source projects, namely, QEMU emulator used for emulation of operating systems and embedded systems hardware; FFmpeg tool for video and audio conversion; TCC (Tiny C Compiler). Some features supported by the engine are: modules, proxies, BigInt and asynchronous generators.

See also:

Sample CMake project

The following sample CMake projects demonstrates how to embed QuickJS engine in a C++ code by using the QuickJSpp C++ wrapper. The CMake script (CMakeLists.txt) downloads the QuickJS source code from its repository and creates static library target for the JavaScript engine which is then linked against the sample application embedding QuickJS. This CMakeLists.txt script fully automates all steps, which relieves the library user from installing QuickJS manually by using GNU make, which is the original building system used by the engine.

GIST Containing all sources:

File: CMakeLists.txt

cmake_minimum_required(VERSION 3.9)
project(QuickJS-Experiment)

#========== Global Configurations =============#
#----------------------------------------------#

set( CMAKE_CXX_STANDARD     17 )
set( CMAKE_VERBOSE_MAKEFILE ON )
set( CMAKE_CXX_EXTENSIONS   OFF)

# ------------ Download CPM CMake Script ----------------#

## Automatically donwload and use module CPM.cmake
file(DOWNLOAD https://raw.githubusercontent.com/TheLartians/CPM.cmake/v0.26.2/cmake/CPM.cmake
                 "${CMAKE_BINARY_DIR}/CPM.cmake")
include("${CMAKE_BINARY_DIR}/CPM.cmake")

#----------- Add dependencies --------------------------#

CPMAddPackage(
    NAME               quickjs 
    GITHUB_REPOSITORY  bellard/quickjs
    GIT_TAG            204682fb87ab9312f0cf81f959ecd181180457bc
    # DOWNLOAD_ONLY YES
    )


# Add this directory where is this file (CMakeLists.txt) to include path. 
include_directories( ${CMAKE_CURRENT_LIST_DIR} )

# =============== QuickJS settings ====================================#

include_directories( ${quickjs_SOURCE_DIR}/ )
message([TRACE] " quickjs source = ${quickjs_SOURCE_DIR} ")

file(GLOB quickjs_hpp ${quickjs_SOURCE_DIR}/*.h )

file(GLOB quickjs_src ${quickjs_SOURCE_DIR}/quickjs.c 
                      ${quickjs_SOURCE_DIR}/libregexp.c 
                      ${quickjs_SOURCE_DIR}/libunicode.c  
                      ${quickjs_SOURCE_DIR}/cutils.c 
                      ${quickjs_SOURCE_DIR}/quickjs-libc.c 
                      ${quickjs_SOURCE_DIR}/libbf.c 
                      )


               add_library( qjs-engine ${quickjs_src} ${quickjs_hpp} )
    target_compile_options( qjs-engine PRIVATE
                                -MMD -MF
                                -Wno-sign-compare 
                                -Wno-missing-field-initializers 
                                -Wundef -Wuninitialized 
                                -Wundef -Wuninitialized -Wwrite-strings -Wchar-subscripts
                          )
target_compile_definitions( qjs-engine PUBLIC 
                                       CONFIG_BIGNUM=y
                                       CONFIG_VERSION="2020-11-08"
                                       _GNU_SOURCE
                           )

if(UNIX)
    target_link_libraries( qjs-engine PRIVATE m pthread dl)
endif()

# =========== Target Settings =========================================#

            # QuickJS compiler. 
            add_executable( qjsc ${quickjs_SOURCE_DIR}/qjsc.c )
target_compile_definitions( qjsc  PUBLIC  CONFIG_BIGNUM=y  CONFIG_VERSION="2020-11-08"  _GNU_SOURCE )            
     target_link_libraries( qjsc  qjs-engine )

            # Sample application that embeds the quickJS Javascript engine. 
       add_executable( main main.cpp   )
target_link_libraries( main qjs-engine )

File: main.cpp

#include <iostream>
#include <quickjspp.hpp>

class ChartXY
{
private:
    double x = 0.0, y = 0.0;
    double width = 100.0, height = 100.0;
public:
    ChartXY()
    { }

    ChartXY(double w, double h): width(w), height(h) 
    { }

    void show() const 
    {
      std::cout << " [ĆhartXY Object] x = " << x << " ; y = " << y 
                << " ; width = " << width << " height = " << height 
                << '\n';
    }

    void set_width(double width) 
    {  
        this->width = width; 
        std::fprintf(stdout, " [ChartXY] Width set to %f \n", width);

    }

    void set_height(double height)
    { 
        this->height = height; 
        std::fprintf(stdout, " [ChartXY] Height set to %f \n", height);        
    }

    double get_height() const { return this->height; }
    double get_width () const { return this->width; }

    void plot_points(std::vector<double> const& points)
    {
        std::cout << " [ChartXY] Plotting points =>> ";
        for(auto p : points) { std::cout << " " << p; }
        std::cout << "\n";
    }
};

qjs::Value
try_eval_module(
             qjs::Context& context
           , qjs::Runtime& runtime
           , std::string const& code)
{
      try
      {
          return context.eval(code, "<eval>", JS_EVAL_TYPE_MODULE);
      } catch( const qjs::exception& ex)
      {
            //js_std_dump_error(ctx);
            auto exc = context.getException();
            std::cerr << (exc.isError() ? "Error: " : "Throw: ") << (std::string)exc << std::endl;
            if((bool)exc["stack"])
                std::cerr << (std::string)exc["stack"] << std::endl;

            js_std_free_handlers(runtime.rt);
            return context.newObject();
      }

}

int main(int argc, char** argv)
{
    std::cout << " [INFO] Started Ok" << std::endl; 

    using namespace qjs;

    Runtime runtime;
    //JSRuntime* rt = runtime.rt;

    Context context(runtime);
    //JSContext* ctx = context.ctx;

    js_std_init_handlers(runtime.rt);

    /* loader for ES6 modules */
    JS_SetModuleLoaderFunc(runtime.rt, nullptr, js_module_loader, nullptr);

    js_std_add_helpers(context.ctx, argc - 1, argv + 1);

    /* system modules */
    js_init_module_std(context.ctx, "std");
    js_init_module_os(context.ctx, "os");

    std::fprintf(stderr, " [TRACE] Before loading code. \n");

    const char* str = R"(
            /*
            import * as std from 'std';
            import * as os from 'os';
            globalThis.std = std;
            globalThis.os = os;
            */

            console.log(" [QUICJS] => =>> Script loaded. Ok. \n");

            for(n = 1; n <= 5; n++){
                console.log(` [QUICKJS-TRACE] n = ${n}/5 `);
            }

            // ----- Define user variables here ----

            asset_path = "/Users/mydir-macosx/data/blackjack.txt";
            game_score = 0.25156;

            let x = 10.352;
            datapoints = [ 0.251, 19.2363, 9.262, 100.125 ];

            console.log(`\n  [QUICKJS] asset_path = ${asset_path}` );
            console.log(`   [QUICKJS] score = ${100.0 * game_score} (in percent) \n`);
            console.log(`   [QUICKJS] data points = ${datapoints} `)
      )";

    try
    {
         context.eval(str); //, "", JS_EVAL_TYPE_MODULE);
    } catch( const qjs::exception& ex)
    {
          //js_std_dump_error(ctx);
          auto exc = context.getException();
          std::cerr << (exc.isError() ? "Error: " : "Throw: ") << (std::string)exc << std::endl;
          if((bool)exc["stack"])
              std::cerr << (std::string)exc["stack"] << std::endl;

          js_std_free_handlers(runtime.rt);
          return 1;
    }

    std::fprintf(stderr, " [TRACE] After loading code. \n");


    int number = (int) context.eval(" 10 * (3 + 1 + 10 ) - 1000 * 2");                               
    std::cout << " [RESULT] number = " << number << '\n';

    std::puts("\n [*] ===== Read configuration variables defined in the js code. ====\n");    
    {
        auto var_asset_path = context.global()["asset_path"].as<std::string>();
        std::cout << "    =>> asset_path = " << var_asset_path << '\n';

        auto score = context.global()["game_score"].as<double>();
        std::cout << "    =>> game_score (%) = " << 100.0 * score << '\n';

        auto points = context.global()["datapoints"].as<std::vector<double>>();
        std::cout << "    ==>> datapoints = [" << points.size() << "]( ";
        for(auto p : points) {  std::cout << p << ' '; }
        std::cout << " ) \n";
    }

    std::puts("\n [*] ===== Define variables in C++-side  ====\n");    
    { 

        context.global()["user_name"]   = context.newValue("Gaius Julius Caesar");
        context.global()["user_points"] = context.newValue(101235);

        auto data = std::vector<std::string>{ "ADA", "RUST", "C++11", "C++17", "C++20"
                                            , "Dlang", "OCaml", "C#(Csharp)" };

        context.global()["user_data"] = context.newValue(data);         

        // Note: This code should be within an exception handler. 
        context.eval(R"(
            console.log(` [STEP 2] user_name = ${user_name} ; points = ${user_points} `);
            console.log(` [STEP 2] user_data = ${user_data} ; type = ${ typeof(user_data) } `);
            console.log(` [STEP 2] user_data[5] = ${ user_data[5] } `)

            // Iterate over the array 
            for(let x in user_data){ console.log(user_data[x]); }
        )");          

    }

    std::puts("\n [*] ===== Register class ChartXY   ====\n");    

    auto& module = context.addModule("chart");
    module.class_<ChartXY>("ChartXY")
      .constructor() 
      .constructor<double, double>()
      .fun<&ChartXY::show>("show")
      .fun<&ChartXY::set_height>("set_height")
      .fun<&ChartXY::set_width>("set_width")
      .fun<&ChartXY::plot_points>("plot_points")
      .property<&ChartXY::get_width,  &ChartXY::set_width>("width")      
      .property<&ChartXY::get_height, &ChartXY::set_height>("height")      
      ;  

    module.add("user_path", "/Users/data/assets/game/score/marks");
    module.add("user_points", 1023523);

    module.function("myfunc", [](double x, double y){ return 4.61 * x + 10 * y * y; });

    const char* module_code = R"(
        import { ChartXY } from "chart";

        import * as chart from "chart"

        console.log(` [SCRIPT] chart.user_path = ${chart.user_path} \n\n`);
        console.log(` [SCRIPT] chart.user_points = ${chart.user_points} \n\n`);

        console.log(` [SCRIPT] Result = ${ chart.myfunc(5.61, 9.821) } \n`);

        let ch = new ChartXY(200, 600);
        ch.show();

        ch.set_width(800.0);
        ch.set_height(700.0)
        ch.show();

        console.log("   [QUICKJS] Change chart dimensions using properties ");
        ch.width = 500;
        ch.height = 660;

        console.log(`\n   <QUICKJS> Chart width = ${ch.width} ; Chart height = ${ch.height} \n`);

        ch.plot_points( [ 10.522, 8.261, -100.24, 7.2532, 56.123, 89.23 ] );
    )";

    try_eval_module(context, runtime, module_code);

    js_std_loop(context.ctx);
    // ----- Shutdown virtual machine ---------------// 
    js_std_free_handlers(runtime.rt);

    return 0;
}

Building and Running

Download sources:

 $ git clone https://gist.github.com/1abdd1d36cd3e973cd1f11f5c20ef7eb qqjs && cd qqjs 

 $ ls
CMakeLists.txt  main.cpp  quickjspp.hpp

Building:

$ cmake -H. -B_build -DCMAKE_BUILD_TYPE=Debug 
$ cmake --build _build --target

Running qjsc (QuickJS - transpiler or C code generator)

 $ _build/qjsc 
QuickJS Compiler version 2020-11-08
usage: qjsc [options] [files]

options are:
-c          only output bytecode in a C file
-e          output main() and bytecode in a C file (default = executable output)
-o output   set the output filename
-N cname    set the C name of the generated data
-m          compile as Javascript module (default=autodetect)
-D module_name         compile a dynamically loaded module or worker
-M module_name[,cname] add initialization code for an external C module
-x          byte swapped output
-p prefix   set the prefix of the generated C names
-S n        set the maximum stack size to 'n' bytes (default=262144)

Running main application, which embeds QuickJS JS engine:

 $ >> _build/main 
 [INFO] Started Ok
 [TRACE] Before loading code. 
 [QUICJS] => =>> Script loaded. Ok. 

 [QUICKJS-TRACE] n = 1/5 
 [QUICKJS-TRACE] n = 2/5 
 [QUICKJS-TRACE] n = 3/5 
 [QUICKJS-TRACE] n = 4/5 
 [QUICKJS-TRACE] n = 5/5 

  [QUICKJS] asset_path = /Users/mydir-macosx/data/blackjack.txt
   [QUICKJS] score = 25.156 (in percent) 

   [QUICKJS] data points = 0.251,19.2363,9.262,100.125 
 [TRACE] After loading code. 
 [RESULT] number = -1860

 [*] ===== Read configuration variables defined in the js code. ====

    =>> asset_path = /Users/mydir-macosx/data/blackjack.txt
    =>> game_score (%) = 25.156
    ==>> datapoints = [4]( 0.251 19.2363 9.262 100.125  ) 

 [*] ===== Define variables in C++-side  ====

 [STEP 2] user_name = Gaius Julius Caesar ; points = 101235 
 [STEP 2] user_data = ADA,RUST,C++11,C++17,C++20,Dlang,OCaml,C#(Csharp) ; type = object 
 [STEP 2] user_data[5] = Dlang 
ADA
RUST
C++11
C++17
C++20
Dlang
OCaml
C#(Csharp)

 [*] ===== Register class ChartXY   ====

 [SCRIPT] chart.user_path = /Users/data/assets/game/score/marks 


 [SCRIPT] chart.user_points = 1023523 


 [SCRIPT] Result = 990.3825099999999 

 [ĆhartXY Object] x = 0 ; y = 0 ; width = 200 height = 600
 [ChartXY] Width set to 800.000000 
 [ChartXY] Height set to 700.000000 
 [ĆhartXY Object] x = 0 ; y = 0 ; width = 800 height = 700
   [QUICKJS] Change chart dimensions using properties 
 [ChartXY] Width set to 500.000000 
 [ChartXY] Height set to 660.000000 

   <QUICKJS> Chart width = 500 ; Chart height = 660 

 [ChartXY] Plotting points =>>  10.522 8.261 -100.24 7.2532 56.123 89.23

1.9 Chaiscript

Scripting engine available as a header-only library that has Javascript-like syntax and easy integration to C++ codebases.

Drawbacks:

  • Header-only => Slow compile-time and large executable size due to the intesive use of templates. Extern templates C++ language feature could reduce the compile time.

Repository:

Web site:

Files

File: CMakeLists.txt

cmake_minimum_required(VERSION 3.0)
project(chaiscript-eval)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_VERBOSE_MAKEFILE ON)

include( FetchContent )

set( BUILD_SAMPLES  OFF CACHE BOOL  "") 
set( BUILD_SAMPLES  OFF CACHE BOOL  "") 
set( RUN_FUZZY_TESTS OFF CACHE BOOL "")
set( RUN_PERFORMANCE_TESTS  OFF CACHE BOOL "")

FetchContent_Declare(
     chaiscript 
     GIT_REPOSITORY  https://github.com/ChaiScript/ChaiScript/
     GIT_TAG         v6.1.0     
)
FetchContent_MakeAvailable(chaiscript)
include_directories( chaiscript-runtime PUBLIC ${chaiscript_SOURCE_DIR}/include )    

add_executable( runner chaiscript-eval.cpp)

if(UNIX)
    target_link_libraries( runner PUBLIC pthread dl )
endif()

File: chaiscript-eval.cpp

#include <iostream> 
#include <string> 

#include <chaiscript/chaiscript.hpp>

void scriptable_function(const std::string& label, int w) 
{
    std::cout << "\n [CALLED] label = " << label << " ; w = " << w << '\n';
}

class Robot
{
    std::string name;
    float x = 0, y = 0;
public:

    Robot(){ }

    Robot(float x, float y){}

    void setPosition(float x, float y)
    {
        this->x = x;
        this->y = y;
        std::fprintf(stderr, " [INFO] Robot moved to x = %f ; y = %f \n", x, y);
    }   

    void showPosition()
    {
      std::fprintf(stderr, " [INFO] Robot position (x = %f, y = %f ) \n", x, y);
    }
};


int main() {

  // Create script engine object 
  chaiscript::ChaiScript chai;

  // Register user function   
  chai.add( chaiscript::fun(&scriptable_function)
          , "scriptable_function");

  chai.add( chaiscript::constructor<Robot()>(), "Robot" );
  chai.add( chaiscript::fun(&Robot::showPosition), "showPosition");
  chai.add( chaiscript::fun(&Robot::setPosition),  "setPosition" );

  const char* code = R"(
        // It supports C++-like syntax 
        for(var i = 0; i < 5; ++i)
        { 
            print(i);
        }

        puts(" ========= Line ================= \n");
        scriptable_function("Moon", 200);
        scriptable_function("Mars", 500);

        var robot = Robot();
        robot.setPosition(200, 400);
        robot.showPosition();

        // User configuration function will be called 
        // by the script engine. 

        def on_init_hook()
        {
            puts("\n [TRACE] User function called. Ok.");
        }

    )";

  // Attempt to evaluate code 
  try { 
      chai.eval(code);
  } catch(chaiscript::exception::eval_error const& ex)
  {
      std::cout << " [TRACE] Exception = " << ex.what() << std::endl;
  }

  std::puts(" ==== Get robot object from script =========");

  auto robot = chai.eval<std::shared_ptr<Robot>>("robot");

  robot->showPosition();
  robot->setPosition(400, 1000);

} // ---- End of main() ----------// 

Check Executable

$ du -h build/runner
23M     build/runner

# Remove debugging symbols 
$ strip build/runner
$ du -h build/runner
7.6M    build/runner

$ file build/runner
build/runner: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2

Running

$ ./runner 
0
1
2
3
4
 ========= Line ================= 

 [CALLED] label = Moon ; w = 200

 [CALLED] label = Mars ; w = 500
 [INFO] Robot moved to x = 200.000000 ; y = 400.000000 
 [INFO] Robot position (x = 200.000000, y = 400.000000 ) 
 ==== Get robot object from script =========
 [INFO] Robot position (x = 200.000000, y = 400.000000 ) 
 [INFO] Robot moved to x = 400.000000 ; y = 1000.000000 

1.10 Python Engine via Pybind11

Documentation:

Advantages:

  • High popularity
  • Lost of libraries
  • Easy usage

Drawbacks:

  • Hard to static link
  • Not ligthweight and designed to be embedded as Lua.
  • It is not possible to run multiple instances of the Python interpreter.
  • It is not possible to sandbox the interpreter and restrict accessing files or process manipulation APIs.
  • Requires pre-installation of Python development headers. But, it can be mitigated by using Anaconda or miniconda Python distributions.

Known Cases:

  • GDB - GNU Debugger
  • IDA Debugger
  • Sublime Text Editor

Sample Project

GIST containing the sources:

File: CMakeLists.txt

cmake_minimum_required(VERSION 3.9)
project(embed-python-scripting)

#========== Global Configurations =============#
#----------------------------------------------#

set(CMAKE_CXX_STANDARD 17)     
set(CMAKE_VERBOSE_MAKEFILE ON)
set(CMAKE_CXX_EXTENSIONS OFF)

# ------------ Download CPM CMake Script ----------------#

## Automatically donwload and use module CPM.cmake
file(DOWNLOAD https://raw.githubusercontent.com/TheLartians/CPM.cmake/v0.26.2/cmake/CPM.cmake
                 "${CMAKE_BINARY_DIR}/CPM.cmake")
include("${CMAKE_BINARY_DIR}/CPM.cmake")

#----------- Add dependencies --------------------------#

find_package(PythonLibs REQUIRED)

CPMAddPackage(
    NAME pybind11 
    URL  https://github.com/pybind/pybind11/archive/v2.5.zip    
    DOWNLOAD_ONLY true  
)

# add_subdirectory( {pybind11_SOURCE_DIR} )
include_directories( ${pybind11_SOURCE_DIR}/include
                     ${PYTHON_INCLUDE_PATH} )

message( [TRACE] "  pybind11_SOURCE_DIR = ${pybind11_SOURCE_DIR} ")

# configure_file(script.py ${CMAKE_BINARY_DIR} COPYONLY )

#----------- Set targets -------------------------------#

add_executable(app1 app1.cpp)
target_link_libraries( app1 ${PYTHON_LIBRARIES} )

add_custom_command(
        TARGET app1 POST_BUILD
        COMMAND ${CMAKE_COMMAND} -E copy
                ${CMAKE_SOURCE_DIR}/script.py 
                ${CMAKE_CURRENT_BINARY_DIR}/script.py)

File: app.cpp

#include <iostream>
#include <string> 
#include <sstream>
#include <fstream>
#include <cassert>
#include <vector> 

#include <pybind11/embed.h>

namespace py = pybind11;

// Requires: <string>, <stream>, <sstream>
std::string readFile(std::string const& file)
{
    auto is = std::ifstream(file);
    if( !is.good() ){
        throw std::runtime_error("Error: stream has errors.");
    }
    std::stringstream ss;
    ss << is.rdbuf();
    return ss.str();
}


int main(int argc, char** argv)
{
    extern std::string pycode;

    auto guard = py::scoped_interpreter{};

    // ------ EXPERIMENT 1 ------------------------------------// 
    std::puts(" [EXPERIMENT 1] ===== Execute Python code ======================\n");

    auto code = readFile("./script.py");

    // std::cout << " code = " << code << "\n";

    auto g = py::globals();

    // Define global variables for the interpreter global scope 
    g["game_assets"] = "/Users/myuser/game_assets";
    g["speed"]       = 20.151;
    g["z_value"]     = 100;    

    // Evaluate Python code
    try {        
        py::exec( code, g );

    } catch(pybind11::error_already_set const& ex) {
        std::cerr << " [PYBIND11 ERROR] " << ex.what() << std::endl;
        return EXIT_FAILURE;
    }

    // ------ EXPERIMENT 2 ------------------------------------// 
    std::puts("\n [EXPERIMENT 2] == Read user defined configuration variables ===\n");

    auto v_x    = g["x"].cast<double>();
    auto v_path = g["path"].cast<std::string>();

    std::cout << " [*]    v_x = " << v_x << std::endl;
    std::cout << " [*] v_path = " << v_path << std::endl;

    return EXIT_SUCCESS;
}

// ----- Internal Python Embedded Module ---------------------------// 


const char* version()
{
    return "SampleModule Version 3.451-ZETA";
}

// Sample "function-object class"
class LinearFunctor
{
public:
    double A = 0, B = 0;

    LinearFunctor();
    LinearFunctor(double a, double b): A(a), B(b){ }

    double GetA() const   { return A; }
    void   SetA(double a) { A = a; }
    double GetB() const   { return B; }
    void   SetB(double b) { B = b; }

    void show() const
    {
        std::cout << " LinearFunction: y(x) = A * x + B" << std::endl;
        std::cout << " => A = " << this->A << " ; B = " << this->B << std::endl;
    }
    std::string toString() const
    {
        std::stringstream ss;
        ss << " LinearFunction: y(x) = A * x + B" << std::endl;
        ss << " => A = " << this->A << " ; B = " << this->B << std::endl;
        return ss.str();
    }
    // Function-call operator
    double operator()(double x)
    {
        return A * x + B;
    }
};

// ---- Internal Module -----------------------// 

PYBIND11_EMBEDDED_MODULE(SampleModule, m) {
    // optional module docstring
    m.doc() = "Sample Python built with C++ CeePlusPlus ";
    m.def("version", &version, "Show Library Version");

    m.def("cppLambda"
          ,[](double x, double y){ return 3.0 * x + y;}
          ,"A C++ lambda object or functor"
          //,py::arg("x"), py::args("y") = 15
    );

    // Register LinearFunction
    py::class_<LinearFunctor>(m, "LinearFunctor")
            .def(py::init<double, double>())             // Register overloaded consructor
            .def("GetA", &LinearFunctor::GetA)            // Reister method GetA()
            .def("GetB", &LinearFunctor::GetB)            // Register method GetB()
            .def("SetA", &LinearFunctor::SetA)            // Reister method GetA()
            .def("SetB", &LinearFunctor::SetB)            // Register method GetB()
            .def("show", &LinearFunctor::show)            // Register method show
            .def("call", &LinearFunctor::operator())      // Register function-call operator with name 'call'
            .def("__call__", &LinearFunctor::operator ()) // Register fun-call operator
            .def("__repr__", &LinearFunctor::toString)    // Register strin representation
            .def_readwrite("A", &LinearFunctor::A)        // Register field A
            .def_readwrite("B", &LinearFunctor::B);       // Register field B

} /** --- End of PYBIND11_MODULE registration --- */

File: script.py

print("   => game_assets = ", game_assets)
print("   =>       speed = ", speed)
print("   =>     z_value = ", z_value)

x: float = 10.0 * 20.51 / 200

path = "C:\\\\Users\\dummy\\Documents\\data"

print(" [PYTHON] The value of x = ", x)

for i in range(5):
    print("   [PYTHON] i = ", i)

# It is not possible to restrict the interpreter!
import os 
print(" [*] Current path = ", os.getcwd() )

print("\n ------------------------------------------")
print("\n =>>> Test Python Internal Module (C++) <<=\n")

import SampleModule as m 
from SampleModule import LinearFunctor 

print(f"    ->   Module Information = [{m.__doc__}] ")
print( "    ->       Module Version = ", m.version())
print( "    -> m.cppLambda(100, 25) = ", m.cppLambda(100, 25) )

functor = LinearFunctor(8.0, -10.0)
print(f"\n C++ Functor -> ${functor} ")

print(" functor(5.0) = ", functor(5.0))
print(" functor(8.0) = ", functor(8.0))

Build and running:

$ cd /tmp 
$ git clone https://gist.github.com/d7fda02034757374a0b0114e54c7daff python-embed-script 
$ cd python-embed-script

$ cmake -H. -B_build -DCMAKE_BUILD_TYPE=Debug
$ cmake --build _build --target 

Program output:

$ _build/app1 

 [EXPERIMENT 1] ===== Execute Python code ======================

   => game_assets =  /Users/myuser/game_assets
   =>       speed =  20.151
   =>     z_value =  100
 [PYTHON] The value of x =  1.0255
   [PYTHON] i =  0
   [PYTHON] i =  1
   [PYTHON] i =  2
   [PYTHON] i =  3
   [PYTHON] i =  4
 [*] Current path =  /home/mxpkf8/temp-projects/python-embed-script

 ------------------------------------------

 =>>> Test Python Internal Module (C++) <<=

    ->   Module Information = [Sample Python built with C++ CeePlusPlus ] 
    ->       Module Version =  SampleModule Version 3.451-ZETA
    -> m.cppLambda(100, 25) =  325.0

 C++ Functor -> $ LinearFunction: y(x) = A * x + B
 => A = 8 ; B = -10

 functor(5.0) =  30.0
 functor(8.0) =  54.0

 [EXPERIMENT 2] == Read user defined configuration variables ===

 [*]    v_x = 1.0255
 [*] v_path = C:\\Users\dummy\Documents\data

Created: 2021-06-04 Fri 15:10

Validate