HeavyThing x86_64 assembler library

Feature Highlights

  • GPLv3 licensed, commercial support available
  • Easy to use, loads of examples
  • Builtin code profiler w/ user interface
  • Native VDSO support
  • Full Unicode support
  • Native 32 bit or 16 bit strings
  • Correct double to string and vs. handling
  • Very fast user-friendly epoll implementation
  • Very fast big integers
  • Full text user interface components and effects
  • Very fast zlib implementation
  • Very fast TLS 1.2 client/server implementation
  • Very fast SSH2 client/server implementation
  • Software and hardware AES
  • SHA3/Keccak,SHA512,SHA384,SHA256,SHA160,MD5
  • HMAC, PBKDF2, scrypt, HMAC_DRBG, Poly1305
  • Curve25519/Ed25519
  • libsodium/NaCl crypto_box compatibility
  • Very fast web client/server implementation
  • ABI compliant

Download

The download link for our HeavyThing library is in the top right of every page on our site, along with the SHA256 sum of the download itself. If you have downloaded the same version and your SHA256 does not match, it has been modified by parties other than ourselves.

Getting Started

First and foremost, you'll need to grab the latest version of flat assembler, and place the fasm binary somewhere in your path. Due to the sheer size of the library, and the conditional compilation employed, you may find it helpful to create an alias to fasm by adding alias fasm='fasm -m 524288' to your .bashrc, though we don't assume the alias exists for our examples.

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/hello_world
$ fasm -m 524288 hello_world.asm && ld -o hello_world hello_world.o
$ ./hello_world
Hello World

Of course the hello world example is silly, but every self-respecting library has to have one. Using the HeavyThing library for such a task is ridiculously overkill. However, a key thing to consider is that unlike nearly every other assembly library, the entirety of HeavyThing is conditionally compiled. This means our hello_world binary does not include every single function in the library:

$ nm hello_world | wc -l
13

Code Profiling

The HeavyThing library includes automatic profiling support, for both user and library code. This is all accomplished with several fasm macros, and can be enabled by modifying the library's default settings located in ht_defaults.inc. The setting you are after is:

	; enable automatic profiling in clocks/calls/callgraph?
	profiling = 0

Changing this setting to 1 and then recompiling will create call graphs and keep track of cycles per call (CPC), total time spent, everything you'd expect from a mature profiler. To make use of the profiler (as well as other library-wide automatic features like framepointers, etc), each function entry needs to start with prolog FUNCNAME and instead of a simple ret, needs to use epilog. There are presently three methods by which to sample/display profiler output: using the TUI (text user interface), as a string, or as JSON.

Of our showcase products bundled with the library, rwasa includes a runtime TUI profiler option. By modifying the aforementioned setting in ht_defaults.inc and recompiling rwasa, we can see the TUI profiler in action by starting it like this:

# ./rwasa -cpu 1 -foreground -sandbox /tmp -bind 80

A note here re: the -cpu 1 startup argument: Since rwasa starts a tui_profiler per worker process, and it attaches itself to your terminal's stdout, chaos would ensue if more than one worker process were allowed to start.

So, upon startup, a tui_profiler appears. The image below reflects all processing that occurred once the tui_profiler was instantiated.

rwasa initial TUI profiler output

When (and only when) profiling is enabled, the library initialisation function ht$init takes some time to determine timing deltas and get the processor out of power management mode (if it is enabled). Because rwasa forks and its child process runs the profiler, the full ht$init is not revealed in the profiler output because the child process calls profiler$reset to clear all of the parent process' counters. What we do see is all of the functions that the child process had to do initially to startup, prior to serving any requests.

Since rwasa is an epoll program, and because the TUI profiler is also epoll based, if we send the R key to the TUI profiler, it will refresh its underlying data with whatever has occurred up to that point in time (most useful for building network applications). A quick note here re: HeavyThing's epoll layer, if you examine epoll.inc, you can also optionally enable profiling the internals of the HeavyThing library's epoll layer. So, if after we start rwasa, we then run webslap against it with the following command:

# ./webslap -n 50000 -c 100 -noui http://127.0.0.1/demo_hook.asmcall

And then send the R key to the TUI profiler for rwasa, we get an updated profiler output that reflects our webslap activity against it:

rwasa post processing TUI profiler output

Our code profiler certainly does add a tremendous amount of overhead to our application, but because of the early delta calculations, it does its best to remove them from the resultant output. We can see that after running our 50,000 requests to it, that 13.873% of the total runtime was spent in epoll$send. This is a direct reflection of the underlying syscall that occurs there and that our actual demo assembly language function hook doesn't do a whole lot.

When we send a space key to the profiler, it switches to callgraph mode. Each nesting level is indicated by the number of leading spaces and its output is listed in the order they were seen (from the commencement of either profiler$init or profiler$reset as in this example). Unlike the top-level overview first screens listed above, the call graph output displays how many calls there were to each function, as well as how many cycles were actually spent exclusively in that function, and then the final column is the total time spent at that function (which includes the time spent in nested function calls from there). Using the arrow keys, we can scroll through our callgraph to find our demo function hook:

rwasa TUI profiler call graph

As noted earlier, the underlying data that makes our profiler's tui_datagrid display can also be dumped as straight text or JSON, which depending on the level of complexity of your application may be easier than making use of the TUI profiler.

Examples

The primary reason that our three showcase pieces sshtalk, webslap and rwasa are bundled and released together with our library is that in our opinion, it doesn't do much good to release a unique set of programming tools without clear and concise evidence of how to use them. To this end, the three products that made it to the showcase and are thus included represent a wide range of HeavyThing's capabilities while still focussing on our target platform being linux x86_64 servers.

The examples that follow represent various features of the HeavyThing library that may be easier to follow than wading through our showcase pieces.

Echo Server

Often the simplest form of a networking library's examples happens to also be the most common starting point for custom server development. This example fires up an INADDR_ANY:8001 echo server listener. Once the following commands are complete, telnet localhost 8001 and fire away.

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/echo
$ fasm -m 524288 echo.asm && ld -o echo echo.o
$ ./echo

Simple Sockets

Similarly, this example performs outbound TCP connections, and will accept either a hostname for DNS lookups, or a numeric address. Optionally a port number can be specified, defaults to 80. This example upon successful socket connect sends a simple HTTP/1.0 GET request and sends the response to stdout:

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/simple_socket
$ fasm -m 524288 simple_socket.asm && ld -o simple_socket simple_socket.o
$ ./simple_socket google.com

Multi-core Echo Server

Similar to the simple echo server, this one will fire up as many processes as desired and is a useful example of multiple-process server handling. Unlike the first example, this one performs simple argument parsing, for the optional -CPUCOUNT and mandatory port number that it listens on.

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/multicore_echo
$ fasm -m 524288 multicore_echo.asm && ld -o multicore_echo multicore_echo.o
$ ./multicore_echo -4 8001

SSH2 Echo Server

Stepping up from the simplistic echo server above, we have not seen an SSH2-based echo server before so we thought it also an excellent and simple example to include. Similar to the above, after these commands are complete, ssh -p 8001 localhost and fire away.

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/sshecho
$ fasm -m 524288 sshecho.asm && ld -o sshecho echo.o
$ ./sshecho

TLS Echo Server

Again similar to our last two examples, only this time making use of TLS for our echo server. After these commands are complete, openssl s_client -connect 127.0.0.1:8001 and fire away

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/tlsecho
$ fasm -m 524288 tlsecho.asm && ld -o tlsecho echo.o
$ # with no arguments, uses rsa_selfsigned.pem, else uses dsa_selfsigned.pem
$ ./tlsecho

TUI Effects Demo

Our HeavyThing library's TUI library is in our opinion the most powerful TUI library ever written for linux. It can be tied to either a terminal session, or an SSH2 server with no user interface code modifications. Any and all TUI components may utilise our effects for transitions, etc. We included a great many "particle" effects that you will see on display with this example. Among them, horizontal and vertical sliding effects, sprinkle, materialise, fountain, gun shot, crumble, vaporise effects are ones we included here.

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/tuieffects
$ fasm -m 524288 tuieffects.asm && ld -o tuieffects tuieffects.o
$ ./tuieffects

TUI fountain effect action shot

TUI Matrix Demo

As mentioned in the source for this effect, c'mon, what self-respecting user interface library would it be without this? This is a simple but effective "The Matrix" movie screensaver, only entirely text mode.

CAUTION: If you are not using something like gnome-terminal, there is a very high probability this will result in your terminal program itself going immediately to 100% CPU. Call this a terminal program killer if you will, but we have experienced significant issues with nearly all non-native-linux terminal programs when running this example. See the full commentary in tui_matrix.inc, but be prepared to kill -TERM something.

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/tuimatrix
$ fasm -m 524288 tuimatrix.asm && ld -o tuimatrix tuimatrix.o
$ ./tuimatrix

TUI Matrix action shot

SHA256

This example computes the SHA256 hash of a given file and outputs the hex to stdout.

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/sha256
$ fasm -m 524288 sha256.asm && ld -o sha256 sha256.o
$ ./sha256 sha256

SHA3

This example is literally a copy-paste-modify of the SHA256 example, only we compute the SHA3/256 instead.

$ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
$ tar xf HeavyThing-1.20.tar.gz
$ cd HeavyThing-1.20/examples/sha3
$ fasm -m 524288 sha3.asm && ld -o sha3 sha3.o
$ ./sha3 sha3
  • libsodium/NaCl crypto_box
  • libsodium/NaCl crypto_box

    This example demonstrates the use of crypto_box_easy and crypto_box_open_easy.

    $ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
    $ tar xf HeavyThing-1.20.tar.gz
    $ cd HeavyThing-1.20/examples/libsodium
    $ fasm -m 524288 libsodium.asm && ld -o libsodium libsodium.o
    $ ./libsodium
    

    Mini-gzip

    $ wget https://2ton.com.au/HeavyThing-1.20.tar.gz
    $ tar xf HeavyThing-1.20.tar.gz
    $ cd HeavyThing-1.20/examples/minigzip
    $ fasm -m 524288 minigzip.asm && ld -o minigzip minigzip.o
    $ ./minigzip minigzip > minigzip.gz
    $ ./minigzip -d minigzip.gz > minigzip.undone
    

    C/C++ Mixing/Integration

    Thanks to client requests, we have added C and C++ integrations with our library, and written a lengthy page explaining the details.

    Application Source Code (as HTML)

    Each of our primary applications and examples source code has been parsed into HTML format for easy perusal from your browser.

    Library Source Code (as HTML)

    Using the search box in the upper right provides an autocomplete method for perusing our library. Note that the string functions indexed for the upper search bar only return the 16 bit functions, as all of the 32 bit function names are duplicates and the index is unique. All of the library include files have been parsed into HTML format, and the master index (so you can browse the source without using the search box) is located below: