ChangeLog

0.11 - 2025-01-31

Added

  • New library: libemilua-main.

  • New library: libemilua-libc-service.

  • byte_span.fill().

  • byte_span.with_zeros().

  • byte_span.first() and byte_span.last().

  • byte_span.inplace_lower() and byte_span.inplace_upper().

  • system.get_lowfd()

  • Module libc_service.

  • Parameter subprocess.libc_service in spawn_vm().

  • Parameter subprocess.source_tree_cache in spawn_vm().

  • Parameter subprocess.native_modules_cache in spawn_vm().

  • Parameter subprocess.ld_library_directories in spawn_vm().

  • Parameter subprocess.pd_daemon in spawn_vm().

  • Parameter module in spawn_vm() accepts filesystem.path too now.

  • Value "\0pid" for the parameter "environment" in system.spawn().

  • Function wait() for acceptors.

  • Property file_descriptor.type.

  • Function filesystem.dev_major(), and filesystem.dev_minor().

  • Function filesystem.open().

  • Function file_descriptor.openat().

  • Function file_descriptor.kcmp().

  • Function file_descriptor.is_socket().

  • Function dup_from() in system.in_, system.out, and system.err.

  • Function system.get_ld_library_directories().

  • init.script

    • dev_major() and dev_minor().

    • caph_cache_tzdata() (FreeBSD).

    • dup() and dup2().

    • close() and closefrom().

    • linkat() and AT_SYMLINK_FOLLOW.

    • bind_unix().

    • access(), eaccess(), and access() flags (F_OK, R_OK, W_OK, and X_OK).

  • More capsicum-related functions.

    • file_descriptor.cap_rights_contains().

    • file_descriptor.cap_rights_remove().

    • file_descriptor.cap_ioctls_get().

    • file_descriptor.cap_fcntls_get().

    • system.caph_limit_stdio() (also in init.script).

Changed

  • byte_span.slice() renamed to byte_span.sub().

  • fiber.interrupt() renamed to fiber.cancel(). Originally Emilua adopted the term “interruption” to adhere to Java and Boost.Thread conventions. Java and Boost.Thread seem to be inspired by EINTR when defining InterruptedException and thread_interrupted. The intention isn’t bad and there’s some logic to it:

    • Send signal to a thread (pthread_sigqueue) to unblock the thread by interrupting the syscall.

    • EINTR is returned from the syscall. If the underlying language has exception support, the error will be translated and communicated in the form of exceptions (so the exception would be EINTR/interrupted).

    • C’s thread cancellation does follow the patterns of an exception mechanism and it’s natural to translate the thread cancellation protocol into the stack unwinding flow that happens when raising exceptions.

    However EINTR isn’t related to thread-state. EINTR is related to an action (which may be tried again by the same thread[1]). EINTR isn’t a sticky state which will come back to bite you in the next action from the same thread (Java even got this semantic wrong). Signal handling per se is already complex enough and full of tricky details to remember. If one is (trying to) studying thread cancellation and stumbles upon signal handling tutorials instead (the situation that can happen if thread cancellation insists in using the same terminology) then the learning process will be needlessly more difficult. It’s of my opinion that one should just avoid mixing the terms together here and just adopt an entirely new term (the way POSIX done when defining thread cancellation…​ not thread interruption).

    One can easily define an exception type whose name is thread_canceled. This exception (no matter the naming chosen) is created, raised and handled…​ by a different — almost self-contained — subsystem than the one defining and handling EINTR errors. It’s okay for this subsystem define a new error type/name just for the thread cancellation process. It’ll end up improving the life of new programmers learning about thread cancellation (which should be a task much more common than handling actual EINTR errors[2]).

    Anyways, over the years, I never really got rid of translating[3] “interruption” as a possibility to interrupt the running code at any step (as in kernel interrupt handlers). So I’d always read code such as my_fiber:interrupt() by superposition both meanings while making some small effort to ignore the new “loaded context” from my mind as it had nothing to do with the problem at hand. fiber:cancel() instead would be really unambiguous and avoid context overload.

  • If called with a directory argument, /usr/bin/emilua will execute the file init.lua inside this directory.

  • spawn_vm(): Passing strings as modules ids to mean a filesystem path in subprocess-based actors no longer work.

  • unix.listen() uses fchmod() instead of umask() so it no longer needs to be called from the master VM to change the socket permission mode bits.

  • SIGPIPE is set to SIG_IGN at process startup. Many Boost.Asio objects won’t use MSG_NOSIGNAL on write() (e.g. asio::posix::stream_descriptor). It’s not realistic to expect every programmer to add extra code to ignore SIGPIPE in every programming project. So let’s just go ahead and migrate to the safer default. Programmers wishing to retain the old behavior can just call system.signal.default(system.signal.SIGPIPE). init.script and PID1 still run with SIGPIPE=SIG_DFL, but even the internal forker service will enjoy the new behavior.

Removed

  • Remove JSON module. It’s now available as a separate plugin.

0.10 - 2024-09-01

Added

  • Function tls.dial().

  • file_descriptor

    • Property non_blocking.

  • New bindings in init.script.

    • fsopen(), FSOPEN_CLOEXEC (Linux).

    • fsmount(), FSMOUNT_CLOEXEC (Linux).

    • move_mount(), MOVE_MOUNT_F_SYMLINKS, MOVE_MOUNT_F_AUTOMOUNTS, MOVE_MOUNT_F_EMPTY_PATH, MOVE_MOUNT_T_SYMLINKS, MOVE_MOUNT_T_AUTOMOUNTS, MOVE_MOUNT_T_EMPTY_PATH, MOVE_MOUNT_SET_GROUP, MOVE_MOUNT_BENEATH (Linux).

    • fsconfig(), FSCONFIG_SET_FLAG, FSCONFIG_SET_STRING, FSCONFIG_SET_BINARY, FSCONFIG_SET_PATH, FSCONFIG_SET_PATH_EMPTY, FSCONFIG_SET_FD, FSCONFIG_CMD_CREATE, FSCONFIG_CMD_RECONFIGURE, FSCONFIG_CMD_CREATE_EXCL (Linux).

    • fspick(), FSPICK_CLOEXEC, FSPICK_SYMLINK_NOFOLLOW, FSPICK_NO_AUTOMOUNT, FSPICK_EMPTY_PATH (Linux).

    • open_tree(), OPEN_TREE_CLONE, OPEN_TREE_CLOEXEC (Linux).

Changed

  • tls.context is now an optional parameter to tls.socket's constructor. If one is not provided, a default per-VM on-first-use generated one will be used.

0.9 - 2024-06-26

Added

  • filesystem.clock.time_point.seconds_since_unix_epoch.

  • New bindings in init.script related to mount_setattr() (Linux).

Changed

  • is_block_file() renamed to is_block_device().

  • is_character_file() renamed to is_character_device().

0.8 - 2024-05-19

Added

  • Add functions dial() and listen() from the likes of Golang.

  • New way of embedding builtin modules to a custom binary/launcher.

Changed

  • The code is now dual-licensed MIT and BSL-1.0. User picks either of these options. The motivation is to make it easier to contribute code back to LuaJIT’s community. Previously it was only easy to contribute code back to the Boost’s community.

  • Split module unix into submodules.

    • unix.datagram_socketunix.datagram.socket.

    • unix.stream_socketunix.stream.socket.

    • unix.stream_acceptorunix.stream.acceptor.

    • unix.seqpacket_socketunix.seqpacket.socket.

    • unix.seqpacket_acceptorunix.seqpacket.acceptor.

  • Removed tables for bit.bor() operations. Flags are now passed as lists of strings.

    • file.open_flag.

    • ip.address_info_flag.

    • ip.message_flag.

    • tls.context_flag.

    • unix.message_flag.

  • Actor messaging is now more asynchronous than before. Emilua intentionally used lots of synchronization points internally for actor messaging as it’d be easier to remove synchronization than to add (if the chosen semantics proved to be wrong later). Fast-forward to the present and it’s clear now that the excessive synchronization is not really useful. The excessive synchronization was not getting in the way for anything, but it wasn’t needed either. The new semantics (channel.send is fully asynchronous to the target actor) are lighter to implement as well so it might benefit some workloads. channel.send still retains some of the previous properties such as most of the error-checking (e.g. detecting channel-closed for many scenarios), post semantics in ASIO-lingo (fiber goes to the end of the execution queue so other fibers have a chance to run), and interruptibility. We could go further and just don’t reschedule the fiber nor check for interruptions at all, but I feel more comfortable doing small gradual changes to see how the changes play out.

0.7 - 2024-04-17

Added

  • Add seccomp support.

  • Add filesystem.mkdir() to complement filesystem.create_directory().

  • filesystem.mode() accepts new arguments now.

  • Add filesystem.chroot().

  • filesystem.current_working_directory() accepts file_descriptor objects on UNIX now.

  • Add extra optional parameter to filesystem.mknod().

  • Add filesystem.clock.epoch(). It’s useful to set the last modification date of every file in some directory for the purposes of a reproducible build or something. However there are more attributes besides last-write-time you need to care about if you’re planning to play with reproducible builds (be warned!).

  • Add filesystem.clock.unix_epoch() and filesystem.clock.now().

  • Add more POSIX bindings to init.script API.

  • Add the flock() family to file.stream and file.random_access.

  • Now it’s possible to configure Landlock mode for the calling process or system.spawn() subprocesses.

  • Add byte_span methods for primitive types serialization (e.g. reading i32le from a 4-sized buffer). It also works as an endianness handling interface. 64-bit integers are omitted from the interface because LuaJIT only offers a hacky way to handle them.

Changed

  • Make subprocess.pid nullable. That’s useful for synchronization when multiple fibers are observing parts of subprocess state.

  • Allow file_descriptor.close() to be called multiple times in a row.

  • Change filesystem.copy_file() parameters.

  • Change every name in the module filesystem from hard_* to hard* (e.g. create_hard_link() to create_hardlink()). This C++17 convention is dumb and Python’s pathlib is the one who got it right.

  • Change default record_separator in stream.scanner to "\n".

  • Always start subprocess-based actors with umask 022.

  • Change system.spawn() parameters from nsenter_* to setns_*.

Fixed

  • Close file descriptors from builtin PID1 so EPIPE propagates sooner.

  • Fix races in filesystem.current_working_directory(). Now fchdir() is used.

  • Small documentation issues.

  • Avoid potential IO double-flush on FreeBSD after fork().

0.6 - 2024-01-06

Added

  • Add FreeBSD’s jails support.

  • Add function format() to format strings. The implementation uses C++'s libfmt.

  • Add more functions to the module filesystem: exists(), is_block_file(), is_character_file(), is_directory(), is_fifo(), is_other(), is_regular_file(), is_socket(), is_symlink(), mode(). It was already possible to query for these attributes. These functions were added as an extra convenience.

  • Add yet more functions to the module filesystem: mkfifo(), mknod(), makedev().

  • New UNIX socket options to retrieve security labels and credentials from the remote process.

  • file_descriptor implemented for Windows pipes and file.stream.

  • Many improvements to Windows version of system.spawn().

Changed

  • Convert decomposition functions from filesystem.path to properties: root_name, root_directory, root_path, relative_path, parent_path, filename, stem, extension.

  • Convert some filesystem.path properties to string: root_name, root_directory, filename, stem, extension.

  • filesystem.path.iterator() will return strings at each iteration now.

Removed

  • Remove HTTP & WebSocket classes. They should be offered as separate plugins.

0.5 - 2023-12-03

Added

  • Add mutex.try_lock().

  • Add module recursive_mutex.

  • Add module future.

  • Add filesystem.chown().

  • Enable IPC-based actors on all UNIX systems.

  • Add Linux Landlock support.

  • Add FreeBSD Capsicum support.

Changed

  • spawn_vm() performs the same module path resolution from require() now. That means it’s possible to use root-imports from spawn_vm().

  • spawn_vm() parameters refactored (API break).

0.4 - 2023-04-03

Added

  • A new byte_span type akin to Go slices is used for IO ops.

  • Actor channels now can transceive file descriptors.

  • Support for Linux namespaces. Now you can set up sandboxes and run isolated actors (or just the well-known containers).

  • Modules ip and tls grew a lot. The API for sockets now supports IO ops on byte_span instances, and plenty of new functions and classes (including UDP) were added.

  • New modules.

    • time: clocks and timers.

    • pipe.

    • unix: UNIX domain sockets.

    • serial_port: serial ports.

    • system: UNIX signals, CLI args, env vars, process credentials, and much more.

    • file: file IO. Only available on systems with proactors (e.g. Windows with IOCP, and Linux with io_uring). BSD can still be supported later (with kqueue + POSIX AIO).

    • filesystem: portable path-manipulation, and plenty of filesystem operations & algorithms.

    • stream: AWK-inspired scanner and common stream algorithms.

    • regex: Basic regex functions. The interface has been inspired by C++, Python and AWK.

    • generic_error: portable error comparison for filesystem, sockets, and much more.

    • asio_error: errors thrown by the asio layer.

    • websocket.

  • Lua programs can define their own error categories now.

  • Several new OS-specific APIs (e.g. Linux capabilities, and Windows' TransmitFile()).

  • Add http.request.upgrade_desired().

  • http.socket can work on top of UNIX domain stream sockets now.

  • Documentation can now be installed as manpages.

  • Support for io_uring.

Changed

  • Upgrade to C++20. The motivating feature for the upgrade was std::atomic<std::weak_ptr<T>>. However, other C++20 features are being used as well.

  • Moved steady_timer to the new module time.

  • tls.ctx renamed to tls.context.

  • inbox.recv() renamed to inbox.receive()

  • Module cond renamed to condition_variable.

  • error_code.cat renamed to error_code.category.

  • spawn_ctx_threads() renamed to spawn_context_threads().

  • inherit_ctx renamed to inherit_context in spawn_vm().

  • Now Emilua is less liberal on accepted values for env var EMILUA_COLORS.

  • Finer-grained cancellation of IO ops.

  • Locales are set at application startup.

  • The build system now makes use of Meson’s wrap system.

Removed

  • Removed println().

  • Removed sleep_for. Its functionality has been replaced by the module time.

  • Removed ip.tcp.resolver. Its functionality has been replaced by ip.get_address_info().

Fixed

  • Bug fixes.

0.3 - 2021-03-04

Added

  • HTTP request and response objects now use read-write locks and there is some limited sharing that you can do with them without stumbling upon EBUSY errors.

  • Improvements to the module system (that’s the main feature for this release). You should be able to use guix as the package manager for your emilua projects.

  • EMILUA_PATH environment variable.

  • Native plugins API (it can be disabled at build configure time).

  • Add logging module.

  • Add manpage.

  • --version CLI arg.

  • Build configure options to disable threading.

Changed

  • Use fmtlib from host system.

0.2 - 2021-01-31

Added

  • Add HTTP query function: http.request.continue_required().

Changed

  • Refactor module system. The new module system is incompatible with the previous one. Please refer to the documentation.

  • Numeric values for error codes changed.

Removed

  • Remove failed_to_load_module error code. Now you should see "iostream error" or other more informative error reasons upon a failed module load.

Fixed

  • Fix build when compiler is GCC.


2. As another example for EINTR plumbing with details difficult to grasp for the newcomer…​ how does one handle EINTR for close()? Should we really be reusing vocabulary that might direct the newcomer to such tutorials that have nothing to do with thread-cancellation and already established some norms for the the use of the term “interrupted”? Why would be wrong to just adopt the alternative proposed POSIX terminology instead?
3. As in…​ internally…​ in my mind…​ automatically/semi-unconsciously.