The world has changed. I’m sure that phrase has been uttered many times before, usually when the world really hasn’t changed. What “back to normal” will look like, no one really knows. At this stage, most of us are under a state of lockdown and has given us to time to think. Although, it should be clarified that this time is perhaps less than we might have expected. In particular for those working from home whilst homeschooling your kids, who are busier than ever.
I’ve tried to use my time as productively as I can over the past few weeks. Like many folks, I’ve ended up baking bread, sometimes successfully, sometimes less successfully. I’ve tried my hand at making a burger, which I confess probably tasted better than it looks, and I’ve already identified ways I think I can improve. When it comes to markets and coding, I’ve tried to learn. Simply observing these very unusual markets in the past few weeks, has been a lesson in itself. I’ve spent the time adding new features and fixes for tcapy, my TCA (transaction cost analysis) Python library, which I’ve recently open sourced, and download it from GitHub. To my knowledge it is the first TCA library which has been open sourced, an attractive alternative to paying hundreds of thousands of dollars on your TCA!
I have also begun to learn to code in q over the past few weeks. q can be used in kdb+/q, a database which is ideally suited to dealing with high frequency tick data and is very fast. Many banks and funds in high frequency trading space use kdb+/q. Furthermore, you can actually do a lot of analytics (or at least the heavy lifting first) inside of kdb+/q on the data directly (as opposed to having to take data out of the database, and doing all analytics in a different language). There are also ways to combine q with Python, which is one of the main reasons I’ve started to learn q. Whilst kdb+/q isn’t open source, there’s also a version of kdb+/q available for free for personal non-commercial purposes if you’re looking to learn the language.
I’ve coded for a while (I won’t specify quite how long!) in many languages. I have also taught Python at banks, funds and at Queen Mary University of London, where I’m a visiting lecturer (if you want me to teach my in-house Python for finance or alternative data workshops at your firm, or via video conference). However, q has always alluded me, aside from a very brief look at it, when I implemented a q wrapper for my tcapy library. As my q references, I have been using the new book Machine Learning and Big Data with kdb+/q by Jan Novonty, Paul Bilokon, Aris Galiotos and Frederic Deleze and Q for Mortals by Jeffry A. Borror – and I’d very much recommend both of these. Paul is also teaching q on the Quantitative Development Certificate (QDC) online course, which will start in a few weeks, and I’ll sure to attend’s Paul’s q online lectures (I’m teaching the Python for finance module as part of the QDC too!). To learn more about the QDC and to sign up, click here.
q is a functional language, unlike Java and Python, which are imperative. The functional nature of q can be a bit difficult to get your head around, if you’re used to imperative languages. q is written on top of the k language, and is somewhat more readable. k itself is a variant of APL. There’s also q-SQL, which looks similar to SQL. q code can often be written in a very terse and concise way, particularly by experienced developers. This can make it difficult to understand, by folks who haven’t written the original code.
What I like about Novotny et. al’s book, is that they’ve tried to strike a balance, and made the code samples accessible to newcomers, with lots of comments and they’ve written it in a modular way. A large part of the book also discusses how to do analytics, for example in q, and not purely the “database” style operations. If code is written in a very terse manner without comments it can end up being difficult to maintain. Even when I’m looking at my own Python code, which is somewhat simpler to understand, if I’ve not written comments and used variable names which are not descriptive, it takes time to work out precisely what I’ve done in the past.
It’ll definitely take me time to master q, but having started, I’ll make sure to keep going. I have to admit though, when it comes to writing q, I’ll probably end up sacrificing conciseness, so my code will probably end up being more verbose with copious comments. However, I think this is probably a worthwhile compromise.