HeavyThing x86_64 assembler library
Feature Highlights
- GPLv3 licensed, commercial support available
- Easy to use, loads of examples
- Builtin code profiler w/ user interface
- Native VDSO support
- Full Unicode support
- Native 32 bit or 16 bit strings
- Correct double to string and vs. handling
- Very fast user-friendly epoll implementation
- Very fast big integers
- Full text user interface components and effects
- Very fast zlib implementation
- Very fast TLS 1.2 client/server implementation
- Very fast SSH2 client/server implementation
- Software and hardware AES
- SHA3/Keccak,SHA512,SHA384,SHA256,SHA160,MD5
- HMAC, PBKDF2, scrypt, HMAC_DRBG, Poly1305
- Curve25519/Ed25519
- libsodium/NaCl crypto_box compatibility
- Very fast web client/server implementation
- ABI compliant
Download
The download link for our HeavyThing library is in the top right of every page on our site, along with the SHA256 sum of the download itself. If you have downloaded the same version and your SHA256 does not match, it has been modified by parties other than ourselves.
Getting Started
First and foremost, you'll need to grab the latest version of flat assembler, and place the fasm binary somewhere in your path. Due to the sheer size of the library, and the conditional compilation employed, you may find it helpful to create an alias to fasm by adding alias fasm='fasm -m 524288' to your .bashrc, though we don't assume the alias exists for our examples.
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/hello_world $ fasm -m 524288 hello_world.asm && ld -o hello_world hello_world.o $ ./hello_world Hello World
Of course the hello world example is silly, but every self-respecting library has to have one. Using the HeavyThing library for such a task is ridiculously overkill. However, a key thing to consider is that unlike nearly every other assembly library, the entirety of HeavyThing is conditionally compiled. This means our hello_world binary does not include every single function in the library:
$ nm hello_world | wc -l 13
Code Profiling
The HeavyThing library includes automatic profiling support, for both user and library code. This is all accomplished with several fasm macros, and can be enabled by modifying the library's default settings located in ht_defaults.inc. The setting you are after is:
; enable automatic profiling in clocks/calls/callgraph? profiling = 0
Changing this setting to 1 and then recompiling will create call graphs and keep track of cycles per call (CPC), total time spent, everything you'd expect from a mature profiler. To make use of the profiler (as well as other library-wide automatic features like framepointers, etc), each function entry needs to start with prolog FUNCNAME and instead of a simple ret, needs to use epilog. There are presently three methods by which to sample/display profiler output: using the TUI (text user interface), as a string, or as JSON.
Of our showcase products bundled with the library, rwasa includes a runtime TUI profiler option. By modifying the aforementioned setting in ht_defaults.inc and recompiling rwasa, we can see the TUI profiler in action by starting it like this:
# ./rwasa -cpu 1 -foreground -sandbox /tmp -bind 80
A note here re: the -cpu 1 startup argument: Since rwasa starts a tui_profiler per worker process, and it attaches itself to your terminal's stdout, chaos would ensue if more than one worker process were allowed to start.
So, upon startup, a tui_profiler appears. The image below reflects all processing that occurred once the tui_profiler was instantiated.
When (and only when) profiling is enabled, the library initialisation function ht$init takes some time to determine timing deltas and get the processor out of power management mode (if it is enabled). Because rwasa forks and its child process runs the profiler, the full ht$init is not revealed in the profiler output because the child process calls profiler$reset to clear all of the parent process' counters. What we do see is all of the functions that the child process had to do initially to startup, prior to serving any requests.
Since rwasa is an epoll program, and because the TUI profiler is also epoll based, if we send the R key to the TUI profiler, it will refresh its underlying data with whatever has occurred up to that point in time (most useful for building network applications). A quick note here re: HeavyThing's epoll layer, if you examine epoll.inc, you can also optionally enable profiling the internals of the HeavyThing library's epoll layer. So, if after we start rwasa, we then run webslap against it with the following command:
# ./webslap -n 50000 -c 100 -noui http://127.0.0.1/demo_hook.asmcall
And then send the R key to the TUI profiler for rwasa, we get an updated profiler output that reflects our webslap activity against it:
Our code profiler certainly does add a tremendous amount of overhead to our application, but because of the early delta calculations, it does its best to remove them from the resultant output. We can see that after running our 50,000 requests to it, that 13.873% of the total runtime was spent in epoll$send. This is a direct reflection of the underlying syscall that occurs there and that our actual demo assembly language function hook doesn't do a whole lot.
When we send a space key to the profiler, it switches to callgraph mode. Each nesting level is indicated by the number of leading spaces and its output is listed in the order they were seen (from the commencement of either profiler$init or profiler$reset as in this example). Unlike the top-level overview first screens listed above, the call graph output displays how many calls there were to each function, as well as how many cycles were actually spent exclusively in that function, and then the final column is the total time spent at that function (which includes the time spent in nested function calls from there). Using the arrow keys, we can scroll through our callgraph to find our demo function hook:
As noted earlier, the underlying data that makes our profiler's tui_datagrid display can also be dumped as straight text or JSON, which depending on the level of complexity of your application may be easier than making use of the TUI profiler.
Examples
The primary reason that our three showcase pieces sshtalk, webslap and rwasa are bundled and released together with our library is that in our opinion, it doesn't do much good to release a unique set of programming tools without clear and concise evidence of how to use them. To this end, the three products that made it to the showcase and are thus included represent a wide range of HeavyThing's capabilities while still focussing on our target platform being linux x86_64 servers.
The examples that follow represent various features of the HeavyThing library that may be easier to follow than wading through our showcase pieces.
Echo Server
Often the simplest form of a networking library's examples happens to also be the most common starting point for custom server development. This example fires up an INADDR_ANY:8001 echo server listener. Once the following commands are complete, telnet localhost 8001 and fire away.
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/echo $ fasm -m 524288 echo.asm && ld -o echo echo.o $ ./echo
Simple Sockets
Similarly, this example performs outbound TCP connections, and will accept either a hostname for DNS lookups, or a numeric address. Optionally a port number can be specified, defaults to 80. This example upon successful socket connect sends a simple HTTP/1.0 GET request and sends the response to stdout:
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/simple_socket $ fasm -m 524288 simple_socket.asm && ld -o simple_socket simple_socket.o $ ./simple_socket google.com
Multi-core Echo Server
Similar to the simple echo server, this one will fire up as many processes as desired and is a useful example of multiple-process server handling. Unlike the first example, this one performs simple argument parsing, for the optional -CPUCOUNT and mandatory port number that it listens on.
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/multicore_echo $ fasm -m 524288 multicore_echo.asm && ld -o multicore_echo multicore_echo.o $ ./multicore_echo -4 8001
SSH2 Echo Server
Stepping up from the simplistic echo server above, we have not seen an SSH2-based echo server before so we thought it also an excellent and simple example to include. Similar to the above, after these commands are complete, ssh -p 8001 localhost and fire away.
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/sshecho $ fasm -m 524288 sshecho.asm && ld -o sshecho echo.o $ ./sshecho
TLS Echo Server
Again similar to our last two examples, only this time making use of TLS for our echo server. After these commands are complete, openssl s_client -connect 127.0.0.1:8001 and fire away
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/tlsecho $ fasm -m 524288 tlsecho.asm && ld -o tlsecho echo.o $ # with no arguments, uses rsa_selfsigned.pem, else uses dsa_selfsigned.pem $ ./tlsecho
TUI Effects Demo
Our HeavyThing library's TUI library is in our opinion the most powerful TUI library ever written for linux. It can be tied to either a terminal session, or an SSH2 server with no user interface code modifications. Any and all TUI components may utilise our effects for transitions, etc. We included a great many "particle" effects that you will see on display with this example. Among them, horizontal and vertical sliding effects, sprinkle, materialise, fountain, gun shot, crumble, vaporise effects are ones we included here.
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/tuieffects $ fasm -m 524288 tuieffects.asm && ld -o tuieffects tuieffects.o $ ./tuieffects
TUI Matrix Demo
As mentioned in the source for this effect, c'mon, what self-respecting user interface library would it be without this? This is a simple but effective "The Matrix" movie screensaver, only entirely text mode.
CAUTION: If you are not using something like gnome-terminal, there is a very high probability this will result in your terminal program itself going immediately to 100% CPU. Call this a terminal program killer if you will, but we have experienced significant issues with nearly all non-native-linux terminal programs when running this example. See the full commentary in tui_matrix.inc, but be prepared to kill -TERM something.
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/tuimatrix $ fasm -m 524288 tuimatrix.asm && ld -o tuimatrix tuimatrix.o $ ./tuimatrix
SHA256
This example computes the SHA256 hash of a given file and outputs the hex to stdout.
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/sha256 $ fasm -m 524288 sha256.asm && ld -o sha256 sha256.o $ ./sha256 sha256
SHA3
This example is literally a copy-paste-modify of the SHA256 example, only we compute the SHA3/256 instead.
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/sha3 $ fasm -m 524288 sha3.asm && ld -o sha3 sha3.o $ ./sha3 sha3
libsodium/NaCl crypto_box
This example demonstrates the use of crypto_box_easy and crypto_box_open_easy.
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/libsodium $ fasm -m 524288 libsodium.asm && ld -o libsodium libsodium.o $ ./libsodium
Mini-gzip
$ wget https://2ton.com.au/HeavyThing-1.24.tar.gz $ tar xf HeavyThing-1.24.tar.gz $ cd HeavyThing-1.24/examples/minigzip $ fasm -m 524288 minigzip.asm && ld -o minigzip minigzip.o $ ./minigzip minigzip > minigzip.gz $ ./minigzip -d minigzip.gz > minigzip.undone
C/C++ Mixing/Integration
Thanks to client requests, we have added C and C++ integrations with our library, and written a lengthy page explaining the details.
Application Source Code (as HTML)
Each of our primary applications and examples source code has been parsed into HTML format for easy perusal from your browser.
- examples/simple_socket/simple_socket.asm317 lines
- examples/echo/echo.asm120 lines
- examples/libsodium/libsodium.asm195 lines
- examples/multicore_echo/multicore_echo.asm247 lines
- examples/sshecho/sshecho.asm183 lines
- examples/tlsecho/tlsecho.asm167 lines
- examples/tuieffects/tuieffects.asm250 lines
- examples/tuimatrix/tuimatrix.asm64 lines
- examples/sha256/sha256.asm98 lines
- examples/sha3/sha3.asm99 lines
- examples/minigzip/minigzip.asm176 lines
- rwasa/rwasa.asm152 lines
- rwasa/arguments.inc840 lines
- rwasa/master.inc337 lines
- rwasa/worker.inc361 lines
- webslap/webslap.asm543 lines
- webslap/globals.inc42 lines
- webslap/master.inc1,984 lines
- webslap/worker.inc824 lines
- webslap/master_ui.inc181 lines
- toplip/toplip.asm3,795 lines
- sshtalk/sshtalk.asm326 lines
- sshtalk/userdb.inc732 lines
- sshtalk/screen.inc2,133 lines
- sshtalk/chatpanel.inc1,030 lines
- sshtalk/chatroom.inc423 lines
- sshtalk/statusbar.inc197 lines
- dhtool/dhtool.asm1,099 lines
- hnwatch/hnwatch.asm63 lines
- hnwatch/eventstream.inc460 lines
- hnwatch/hnmodel.inc703 lines
- hnwatch/textify.inc221 lines
- hnwatch/ui.inc1,724 lines
Library Source Code (as HTML)
Using the search box in the upper right provides an autocomplete method for perusing our library. Note that the string functions indexed for the upper search bar only return the 16 bit functions, as all of the 32 bit function names are duplicates and the index is unique. All of the library include files have been parsed into HTML format, and the master index (so you can browse the source without using the search box) is located below:
- aes.inc1,423 lines
- aesxts.inc233 lines
- align_macros.inc162 lines
- base64url_latin1.inc452 lines
- base64_latin1.inc451 lines
- bigint.inc10,923 lines
- blacklist.inc295 lines
- breakpoint.inc27 lines
- buffer.inc1,337 lines
- call.inc82 lines
- ChangeLog652 lines
- cleartext.inc263 lines
- cookiejar.inc942 lines
- crc.inc500 lines
- curve25519.inc5,800 lines
- dataseg_macros.inc78 lines
- date.inc1,336 lines
- debug_macros.inc42 lines
- dh_groups.inc44 lines
- dh_pool.inc75 lines
- dh_pool_16k.inc133 lines
- dh_pool_2k.inc157 lines
- dh_pool_3k.inc157 lines
- dh_pool_4k.inc157 lines
- dh_pool_6k.inc157 lines
- dh_pool_8k.inc157 lines
- dir.inc148 lines
- ed25519.inc10,253 lines
- epoll.inc3,512 lines
- epoll_child.inc167 lines
- epoll_dns.inc1,277 lines
- fcgiclient.inc911 lines
- file.inc354 lines
- fileshadow.inc2,444 lines
- formatter.inc2,157 lines
- heap.inc1,114 lines
- hmac.inc821 lines
- hmac_drbg.inc570 lines
- ht.inc673 lines
- htcrypt.inc731 lines
- http1.inc837 lines
- httpheaders.inc3,750 lines
- htxts.inc178 lines
- ht_data.inc36 lines
- ht_defaults.inc523 lines
- io.inc273 lines
- json.inc1,646 lines
- list.inc874 lines
- mapped.inc455 lines
- mappedheap.inc621 lines
- maps.inc4,544 lines
- math.inc996 lines
- maxmind.inc592 lines
- md5.inc531 lines
- memfuncs.inc1,822 lines
- mimelike.inc4,037 lines
- mtree.inc3,293 lines
- mtree_aesxts.inc370 lines
- pbkdf2.inc298 lines
- png.inc785 lines
- poly1305.inc1,126 lines
- privmapped.inc331 lines
- profiler.inc1,315 lines
- rdtsc.inc62 lines
- rng.inc1,047 lines
- scrypt.inc567 lines
- sha1.inc677 lines
- sha2.inc2,146 lines
- sha3.inc580 lines
- sleeps.inc55 lines
- sodium_compat.inc1,568 lines
- ssh.inc6,013 lines
- string16.inc5,023 lines
- string32.inc5,076 lines
- string_math.inc2,158 lines
- syscall.inc341 lines
- sysinfo.inc71 lines
- syslog.inc300 lines
- tls.inc6,875 lines
- tui_alert.inc519 lines
- tui_ansi.inc1,212 lines
- tui_background.inc263 lines
- tui_bell.inc165 lines
- tui_button.inc294 lines
- tui_datagrid.inc408 lines
- tui_effect.inc1,013 lines
- tui_effects.inc1,036 lines
- tui_form.inc780 lines
- tui_geometry.inc484 lines
- tui_gridguts.inc1,073 lines
- tui_label.inc789 lines
- tui_lines.inc112 lines
- tui_lock.inc261 lines
- tui_matrix.inc676 lines
- tui_newsticker.inc315 lines
- tui_object.inc3,095 lines
- tui_panel.inc641 lines
- tui_png.inc537 lines
- tui_progressbar.inc389 lines
- tui_progressbox.inc216 lines
- tui_render.inc1,782 lines
- tui_simpleauth.inc1,960 lines
- tui_spacers.inc110 lines
- tui_spinner.inc144 lines
- tui_splash.inc502 lines
- tui_ssh.inc635 lines
- tui_statusbar.inc443 lines
- tui_terminal.inc1,024 lines
- tui_text.inc3,903 lines
- tui_textbox.inc229 lines
- tui_typist.inc627 lines
- unicodecase.inc947 lines
- url.inc1,778 lines
- vdso.inc240 lines
- vector.inc894 lines
- webclient.inc2,042 lines
- webserver.inc6,553 lines
- X509.inc4,133 lines
- xmlmemnode.inc2,439 lines
- xmlparser.inc4,508 lines
- zlib_deflate.inc4,805 lines
- zlib_inflate.inc2,656 lines