Alright .. Yes the title feels bombastic.

I’m not here to freak you out 😂 but I do want to share an important POV that may affect your process when it comes to working with data in financial markets.

Coding is great, don’t get me wrong…

Let me start by saying that knowing how to-code is extremely powerful, I’m not here to bash coding at all. In fact — I LOVE IT! (am a novice coder myself 🤓) it’s even more powerful when you utilize it in the right way at the right time👌 , But for certain operations or processes it just does not make any sense. Programming is a way to automate repetitive tasks that are cumbersome with a set goal in mind to make the process more efficient or faster. Whenever you reach that messy path where writing code becomes longer, more complicated, requires more resources, etc’ than the normal alternative — that’s when you should ask yourself :

is coding really necessary here ? will it make my end goal easier or harder to implement ? will it consume more or less of my time ?

A good general rule should be:

Coding my program < doing it manually over time OR using other software

You are a trader, not a coder!

As a trader or investor — you don’t want to start learning how to code. Your job is hard enough being a trader, analyzing the markets and coming up with investable ideas. You want to spend your entire time maximizing the things you are strong at and leveraging them, as oppose to start taking coding classes and learning a new language. YOU HAVE NO TIME FOR THAT IN 2022. You will be wasting a lot of time on learning coding instead of coming up with new strategies and ideas. That time will cost you money in the long run — so you now will be spending time, money, and maybe even missing opportunities so we have the opportunity cost here as a factor as well.

Getting financial data with an API is HARD!

Even if you are fortunate enough to know programming I would still advise not to go with the route of pulling data with code. sounds crazy ? Wait, hear me out before calling me names 🙈 ➡️

Because it’s extremely hard, in-efficient and sometimes impossible to do so.

Say you want to filter the entire market for some pattern, for example you want to say “Give me the all stocks that went up in the past year more than 100%”.

If you want to filter the market to get data based on different conditions you have in mind — you will need the entire universe of stocks at your disposal, as how will you get all the actual candidates without missing any, if you are not scanning the entire market, each time ? so that means you will have to somehow pull with code the entire database of stocks. that is a really hard process to do. you, that just wanted to pull stocks with a simple, certain pattern, now needs to build an entire local database of stocks ? that will take you months , if not years. In addition to that — no API vendor will just gladly “give you” their entire database to download, they more often than not put in place restrictions and limit your API calls so that you can’t bluntly just pull their entire databases.

And even in the highly unlikely event you were able to download the entire database now you need to :

• clean it
• normalize it
• maintain it
• pay for servers to store it
• update it

Filtering through your data, you have no idea what you got yourself into —

Ok, say you were able to somehow get the API vendor to give you all their databases. Say you were able to build an entire server infrastructure to hold the data, clean it, etc’. your’e spending thousands of dollars a month. NOW — you will have to create an advanced filtering system to sift through the data and query it as you please, right ?

We are here to build strategies and test new ideas. We have no use in a huge, complicated database if we can’t query it based on our needs or extract the data as we please, right ?

if we take our previous example of pulling all 100% gainers in the past year — we need a way to pull that from our un-necessary, locally stored db. you will find that it is very hard to build such a filtering system, that we can easily find our way through and pull the data that we need (tip: I’ve done it so trust me on that 😜)

When working with large amounts of data (we will be working with the entire universe of US stocks after all) — it’s very hard to come up with fast, efficient query speed times, that will return to us the data that is relevant for our strategy only. You will have to be a master in SQL. You will have to become a master in Databases. Correct me if I’m wrong but, is that what you signed up for ? becoming a db expert, or trading the markets ?

What if you have many different strategies you need to pull data for ? that’s more SQL skills to develop and more messing around with data and building coding solutions. It’s a clear path to spend your time on building a financial database, instead of building profitable trading strategies ! (Watch out going down this rabbit hole 🌀)

I’m going to ask this frighting question, but please don’t freak out — what if…… you missed some of the data while initially downloading it 😮 ??? do you realize how much work you would waste while trying to re-download the data again, as oppose to just using a solution that does all of this for you ?

Additional complexities like splits and delisted securities

on top of all the difficulties mentioned above — there are extra steps to take when working with financial data, lots our specified here: why-it-is-important-to-work-with-high-quality-financial-data

but to name a few :

• You will have to incorporate splits and dividends to your OHLCV data to get sufficient adjusted and unadjusted data
• You will have to take care of delisted securities
• You will have to aggregate data correctly from raw data to OHLCV data

all of these would need to be taken by you, since you are building a local db, or are pulling raw data from an API that has no context, no cleaning, no splits, etc’.

Another thing to think about is the more data you are using the more tools and data vendors you will have to use. If you start using news in your data, you will have to adjust and build tools for that, if you start trading options instead of equities you will have to maintain that as well > it’s just more work in the future for you 🙁

There is a Solution for all of this

spikeet.com was created to solve this problem exactly.

If you are a trader and are looking for financial data, without needing to learn how-to-code, without needing to download and maintain an entire universe of stocks or databases, and if you want a system that you can easily query with plain words and run simple & complex data scans — then you must give it a go.

It will save you loads of time, you won’t need to learn how to code, you won’t need to maintain your own servers, and you will be using a highly advanced data platform that sources all of it’s data from the exchanges and has very high quality datasets.

We also have build-in templates you can easily use and run :

We have also build a custom formula text box, where you can create complex formulas or conditions to query the data:

So do yourself a favor, and stop using old school methods to work with financial market data — instead take advantage of our innovative data platform — it will save you loads of time and money in the long run !

Noam — A retired developer that used to get data with API but switched to Spikeet 😊📊