I first encountered q/kdb+ at a quant job in 2007. I learned so much from the array semantics about how to concisely represent time-series logic that I can't imagine ever using a scalar language for research.
Fun fact: the aj (asof join) function was my inspiration for pandas.merge_asof. I added the extra parameters (direction, tolerance, allow_exact_matches) because of the limitations I kept hitting in kdb.
The aj function at its heart is a bin (https://code.kx.com/q/ref/bin/) search between the two tables, on the requested columns, to find the indices of the right table to zip onto the left table.
aj[`sym`time;t;q]
becomes
t,'(`sym`time _q)(`sym`time#q)bin`sym`time#t
The rest of the aj function internals are there to handle edge cases, handling missing columns and options for filling nulls.
A lot of the joins can be distilled to the core operators/functions in a similar manner. For example the plus-join is
I couldn't figure-out how Arthur's bin matched on symbol though, so I switched to a linear scan on the right table to record the last-seen index for each "by" element. While it worked, my hash table was messy because I relied on Python to handle a whole tuple as a key, which had some issues during initial testing.
The asof join I wrote for Empirical properly categorizes the keys before they are matched. That approach worked far better.
K fucking rules if you're trying to minmax a game or balance your own.
Figuring out things like "what percentage of the time will my starting hand contain the cards I need for my deck to function, and if it doesn't, how many mulligans will it take" will basically ruin competitive MTG for you. I used to buy physical verisons of the decks I made it to top 500 with in Arena, stopped because it wasn't a fun challenge anymore (and lost all interest when they started dragging other IPs in, they'll never get another cent from me)
I first encountered q/kdb+ at a quant job in 2007. I learned so much from the array semantics about how to concisely represent time-series logic that I can't imagine ever using a scalar language for research.
Fun fact: the aj (asof join) function was my inspiration for pandas.merge_asof. I added the extra parameters (direction, tolerance, allow_exact_matches) because of the limitations I kept hitting in kdb.
https://code.kx.com/q/ref/aj/
https://pandas.pydata.org/docs/reference/api/pandas.merge_as...
The aj function at its heart is a bin (https://code.kx.com/q/ref/bin/) search between the two tables, on the requested columns, to find the indices of the right table to zip onto the left table.
becomes The rest of the aj function internals are there to handle edge cases, handling missing columns and options for filling nulls.A lot of the joins can be distilled to the core operators/functions in a similar manner. For example the plus-join is
Indeed, my very first attempt used numpy.searchsorted:
https://numpy.org/doc/2.2/reference/generated/numpy.searchso...
I couldn't figure-out how Arthur's bin matched on symbol though, so I switched to a linear scan on the right table to record the last-seen index for each "by" element. While it worked, my hash table was messy because I relied on Python to handle a whole tuple as a key, which had some issues during initial testing.
The asof join I wrote for Empirical properly categorizes the keys before they are matched. That approach worked far better.
https://www.empirical-soft.com/tutorial.html#dataframes
Similarly, this is how it was introduced in ClickHouse in 2019: https://github.com/ClickHouse/ClickHouse/pull/4774
I wish there was a language like K that worked with single precision floats. Would be great to use with graphics.
q is like k and has single precision floats.
q is closed source so it isn't really viable to build anything out of it unfortunately. I wonder how they made floats work since it is based on k.
the github repo linked, under "manual.md", has a link which redirects to malware, such as that ostensibly to "k6".
K fucking rules if you're trying to minmax a game or balance your own.
Figuring out things like "what percentage of the time will my starting hand contain the cards I need for my deck to function, and if it doesn't, how many mulligans will it take" will basically ruin competitive MTG for you. I used to buy physical verisons of the decks I made it to top 500 with in Arena, stopped because it wasn't a fun challenge anymore (and lost all interest when they started dragging other IPs in, they'll never get another cent from me)