Getting OCaml running on the ESP32
I was looking for some small Christmas stocking fillers to give to techie friends and decided to try to find some interesting electronics boards from China.
In the end, I went with the WEMOS Lolin32 Lite which features Espressif's ESP32. If you're not familiar with the ESP32, it's an awesome little chip that features the following:
- Dual core 240Mhz 32-bit Xtensa LX6s
- Wi-Fi (802.11 b/g/n) and Bluetooth (v4.2 + BLE)
- 520kb of SRAM
- A separate ultra-low power processor
The Lolin32 Lite couples that with 4mb of flash, micro-usb connection and Li-Po charging circuitry.
Espressif has a gcc-based toolchain and an "IoT Development Framework" which provides a port of Newlib, FreeRTOS, LWIP and a whole host of other frameworks.
My friend Anil suggested that with a gcc and libc, porting of the OCaml interpreter would be fairly easy. He was mostly right.
Caml runtime
It took a little while to understand how OCaml's build system worked and thankfully it seems there's been good support for cross compilation since 4.02. The configure script has a pretty funky way of determining features of the compiler and runtime by compiling lots of small C programs and seeing what builds. This required a few small changes where features were detected but only partially available via Espressif's port of Newlib. Posix signals and BSD sockets were two cases where this happened. I should expand the code for the tests to cover the missing functionality and try to upstream it, which would avoid the configure script hacks.
Rebuilding Newlib
Unfortunately either I was failing or Espressif's build of newlib doesn't seem to include signal(). This meant a rebuild of newlib was required without the SIGNAL_PROVIDED flag, which includes an implementation of signal(). There's also no support for directories in the IDF, so I had to stub out some parts of sys.c and unix.c. With those changes it was possible to get libcamlrun.a compiled.
Once I had a cross compiled bytecode runtime, I was most of the way there.
Building an image
Next step was to get some OCaml compiled which could then be incorporated into the image to flash. This is actually pretty simple with ocamlc and custom runtimes and you end up with a C source file you can then throw in to the rest of the IDF component build system. I wrote a little bit of C that kickstarted ocaml via caml_startup and had a buildable image to flash.
Debugging
I flashed the board and immediately got an abort after malloc failed. First things to tweak were the garbage collection settings which were not designed for 512kb of ram. I tuned many of those and was still getting an abort but after some instrumentation it turns out that the runtime allocates a 64kb buffer for both stdin and stdout. After reducing those buffers considerably, the interpreter no longer aborted! It didn't, however, print anything out - which concerned me.
After a fair amount of debugging, I still have no idea where stdout goes. It's certainly not the same place as printf, which makes it to the monitor. Once I had that figured out, I realised I had a functioning interpreter!
State of play
There's a Dockerfile for the whole build process:
- Installs the prerequisites, Xtensa gcc port, Espressif IDF
- Rebuilds Newlib
- Installs an OCaml via OPAM, then builds the OCaml ESP32 bytecode runtime
- Finally builds a simple Hello World OCaml project and builds an image
You should be able to then flash the resulting image with make flash if you have a dev board connected and have passed the USB-serial device through to the container with --device=/dev/ttyUSB0 (on Linux).
Short term TODOs
There are a couple of TODOs that probably need to be cleaned up or fixed. As I mentioned earlier, we could expand some of the hasgot tests to include functionality Espressif's Newlib build doesn't have and this would simplify some of the configure changes. Figuring out how to redirect stdout and stderr to the monitor would also be incredibly useful.
Longer term plans
A native compiler backend for Xtensa would mean we could produce a more compact and hopefully more performant image which would be very useful in environments with tight power budgets. Speaking of low-power, some kind of DSL for programming the ultra-low power core on the board would also probably be very useful.
In terms of networking, the Espressif IDF ships with an lwip port for networking but there are sufficiently low level interfaces available for the Wi-Fi device that could work with Mirage's tcpip direct driver.
Any volunteers?
End result
