Temporal Analysis LLC
It's Amazing What Computers Can Do These Days
Joining the "uncloud" revolution - a hands-on approach to data engineering and big data systems.
Why on-prem (“uncloud”)
Speed the learning loop
Why have I built my own infrastructure instead of using cloud services?
- Psychology: Ever run a job that utilized all cluster resources for a week and errored out? That would cost $500 for equivalent cloud compute. Having already paid for the hardware makes it easier try things out - and shrug off mistakes.
- New Skills: RAM/CPU selection, cooling, storage media, distributed filesystems, networking, Linux tuning, scheduling, provisioning.
- Cost Control: No surprise bills. Predictable monthly power and networking costs.
Modern OSS Rules!
Thank Goodness for the Apache Foundation
When building any system it's often best to start small and see what you can do. I started with some Python that could poll prices and issue HTTP requests to place orders. It worked for a single symbol, but wouldn't scale.
I've been pulling apart that MVP into more robust services and learning how to keep all the pieces working together.
There's a lot of power in combining streaming data with batch processing, and the Apache ecosystem has a lot of great tools to help make that happen.