All posts
-
Running TigerBeetle without a control plane database. Part two.
Part one and some articles in between.
-
Running TigerBeetle without a control plane database. Locking.
Of course, TigerBeetle does not provide anything for locking, but I don’t want to introduce a control-plane database or add Redis to the mix, so let’s build some new nails for the TigerBeetle hammer.
-
“AI” is the fast food of software development.
Yes, it fills your stomach with calories quickly and cheaply. No, a few fast food meals will not kill you.
-
Could not keep it under the rug until April Fool’s Day
-
Running TigerBeetle without a control plane database. Part one.
TigerBeetle is a database built for financial accounting, and the only record types available are Accounts and Transfers. That might be enough for the simplest accounting setup, but not for any rea...
-
I am a huge fan of System V Inter Process Communication primitives. There is some rawness and UNIX spirit to them. There is a newer and kinda “improved” version of those primitives named POSIX IPC....
-
Talking to payment cards over NFC
I had a great experience speaking about contactless payment cards at BSides Krakow. For those who want to get their hands dirty:
-
We had a requirement to display the last N transactions on the ATM screen (”mini-statement”). The simplest solution is to keep the list of transactions, order them by date, and take the newest N tr...
-
Tracking Gunicorn's busy worker count
I was investigating performance issues of a Django application running with Gunicorn behind a Nginx server. First, I added more timing information to Nginx access.log:
-
Many go to Martin Fowler for microservice architecture, distributed systems, micro frontends, event sourcing, and other fancy ideas about architecture, but a few have noticed the advice to do a mon...
-
Transactional task outbox in Django with django-taskq
We have given up on distributed transactions (2PC) but have not given up working with multiple resources like the database, message brokers, and queues. Instead, everybody tries to build their own ...
-
A Philosophy of Software Design
In a world full of Uncles Bob be John Ousterhout
-
Serializable isolation level and transaction processing
While on the topic of On-Line Transaction Processing Benchmarks, it’s interesting to observe the strategies companies employ to achieve optimal results. All code for both the transaction monitor an...
-
A broker-less distributed messaging system from the previous century
When examining the On-Line Transaction Processing Benchmark, most people focus on the performance numbers and the database software. But there is another column named “TP Monitor” that lists the tr...
-
My Friday's "old man yells at cloud" moment
Casey Muratori had this to say:
-
Python JSON encoder apples and oranges and making it faster
There are several JSON library benchmarks like this one where the built-in Python json library is not the worst performer but there are several libraries that have even up to 4x better result. Some...
-
Ledgers are simple, stop spreading FUD!
When you do need thousands and tens of thousands of transactions per second, TigerBeetle is there for you. A few hundred per second you can do it yourself in ~25 lines of code. And you can reach th...
-
Making sense of card range query EXPLAIN plans
I was listening to a talk about finding a PAN in a haystack and this slide came up with performance numbers in seconds:
-
The COST of double-entry accounting
Sometimes I think the most important paper of our times is Scalability! But at what COST?We build “web-scale” systems and get excited about how scalable they are and how we solve the problems of sc...
-
bpftrace username and the openat system call
Once again I am playing with eBPF and bpftrace. This time I am trying to trace all file access. Whenever a file is open, created, or deleted I want to print the filename, the process ID, and the us...
-
Wikipedia says that O stands for:
-
So many people throw SOLID around as the unquestionable truth that I decided to dig a bit deeper and look into its origins. I will start with S.
-
The correct way to list the queues on the Linux machine is by using the ipcs -q command. However, I have not seen anything in the system calls that would provide such a capability. But Today I Lear...
-
Monitoring Oracle Tuxedo with Linux eBPF
I have built several scripts for monitoring Oracle Tuxedo over the years. Most relied on the tmtrace and tputrace(3c) features provided by Tuxedo itself. I also tried to collect statistics on the I...
-
Optimizing Python application's Docker image with strip
I was going through some Docker images of large applications and looking at the layer sizes with dive. A lot of space was used by installed dependencies under /usr/local/lib/python3.10/dist-package...
-
Accounting systems before TigerBeetle
TigerBeetle is an interesting project to follow. It links to interesting papers, it challenges assumptions, algorithms, and architecture of past systems. It satisfies my cravings for low-level pro...
-
Today I Learned... WTF, Kafka?
I was reading about offset retention and was mentally prepared for losing consumer offsets when the consumer has been offline for 7+ days. But I realized that it’s 7+ days since the last update of ...
-
Open your conftest.py and type the following lines:
-
How I made `json.dumps` ~20% faster
My journey to understand the performance and concurrency of FastAPI services lead me to the Python json.dumps function producing JSON string out of object tree. And I ended up making it a bit faster:
-
FastAPI and cooperative multi-threading pt. 2
After finding a good enough solution for FastAPI and cooperative multi-threading issues, a part of me was still not happy with the results. There was a significant drop in the number of concurrent ...
-
FastAPI and cooperative multi-threading
Cal Paterson wrote a great article comparing and describing synchronous and asynchronous Python frameworks and explaining whyasynchronous frameworks go a bit wobbly under load. This is a story of h...
-
Optimizing Kafka producers for latency
TL;DR Don’t forget to set socket.nagle.disable=True to disable Nagle’s algorithm
-
I have a piece of C++ code that calls user-defined functions implemented in Python. Instead of requiring all functions to have the same signature with 6 arguments, the C++ code inspects the functi...
-
The hitchhiker's guide to the tpcall flags
Oracle Tuxedo documentation of the tpcall function describes (mostly) the high-level behavior of the function. But in addition to that, I am interested in how the messages will be sent through the ...
-
ltrace is a tool for tracing dynamic library calls. I use it from time to time and this time I needed to trace the msgrcv system call. Usually, you would use strace to trace the system calls, but I...
-
Fchg32 is the Swiss Army knife of Tuxedo FML32
FML32 is Oracle Tuxedo buffer type similar to std::multimap in C++ or multidict in Python where the key might occur more than once in the container. Fchg32 is the function for changing a value for ...
-
Debugging Boolean Expressions of Fielded Buffers
I described the SIGFPE bomb of Boolean Expressions before. Going through the list of the C functions I was reminded of the Fboolpr32 function that prints the expression tree as it was parsed. Ten m...
-
Making sense of Tuxedo's SCANUNIT, SANITYSCAN, and BLOCKTIME
Time accounting is strange in Oracle Tuxedo. First, there is the SCANUNIT parameter which must be a value between 0 and 60 seconds and also a multiple of 2 or 5 (10 by default). Then the SANITYSCAN...
-
Servers in Oracle Tuxedo can be configured either in Multiple Servers - Single Queue (MSSQ) setup or Single Server - Single Queue (SSSQ) mode. More in-detail information about SSSQ and MSSQ setup c...
-
Boolean Expressions of Fielded Buffers
Oracle Tuxedo has the Fboolev32 function for evaluating Boolean expressions in which the “variables” are the values of fields. The expressions are a subset of the C programming language with a nice...
-
Tracking Oracle Tuxedo file transfer
Oracle Tuxedo uses System V IPC message queues for sending messages between processes. These queues live inside the OS kernel and are limited in size:
-
A Prometheus exporter of Tuxedo metrics
Now that my book about Oracle Tuxedo is completed, I can work on some code that did not make it into the book. Here is the first by-product: a Prometheus exporter of Tuxedo metrics. Chapter 6 (amon...
-
Oracle Tuxedo queues illustrated
I did investigate tpacall() before and you can find more details there.
-
A server waits for a new incoming service call by using msgrcv() system call on IPC queue. The call blocks until a message is available or a signal interrupts the system call.
-
tpacall(3c) and XA transactions
Oracle Tuxedo allows us to develop transactional service-oriented (or microservice) applications easily and performance is quite good. So easy and performant that I would not care about implementin...
-
There’s already a collection of articles describing problems others have with Go: Go is not good
-
Today I Learned one more thing for “tabs versus spaces” debate: Java hates tabs.
-
tpadvertisex(3c): a new cool Oracle Tuxedo 12.2 feature that works
I wrote before about tpadvertisex() and how it did not work for me
-
tpadvertisex(3c): a new cool Oracle Tuxedo 12.2 feature
A service call in Oracle Tuxedo ends up in some System V message queue. What happens next depends on the configuration in UBBCONFIG:In a Single-Server-Single-Queue setup exactly one instance of the...
-
Oracle Tuxedo internals - blocking `tpgetrply()`
This post is a follow-up for blocking tpacall()
-
Oracle Tuxedo internals - blocking `tpacall()`
A service call in Oracle Tuxedo is performed by the tpcall() function that actually is a wrapper around two other functions:
-
How many PINs your payment card has?
First we have to talk about how payment card PIN (Personal Identification Number) is verified. PIN can verified “offline” by the software running on the chip if you have a chip card and are paying ...
-
This started out as a Hacker News comment about a blog post about select being fundamentally broken After that the same author wrote why epoll is broken as well. I don’t know what problem author is...
-
Changing model of time and space
The morning paper has this epic piece today that I’ll save here:
-
DIY Tuxedo Monitoring - ULOG and TMTRACE
Going the least painfull way I wrote a Python script that analyzes Oracle Tuxedo user log entries and extracts trace information from it. It’s pretty easy to do, does not require compilation of som...
-
GCC implementation of standard C++ library used to have Copy-On-Write std::string implementation. It had the benefit of making string copies fast but it also made strings hard to use in multi-threa...
-
DIY Tuxedo Monitoring - Collecting information
At one point I started to build a monitoring solution for Oracle Tuxedo so these are my notes along the way. The best way to start is reading Monitoring Your Tuxedo Application.