Understanding Data Sources
To gain the most clarity for making data-driven decisions, it is expedient to work with structured data sources.
It's best to organize your data in-house before you start importing any feeds. When you connect to data sources like a list of countries, states, cities and even a phone carrier database, you would need to organize this data in-house so you can easily use it. In-house refers to your own local and cloud storage systems.
While it is good to locally host your data for faster research, cloud backup is important. Yesterday morning, my PC crashed and I lost everything on it but here I am writing for this post.
The business data for all my companies were backed up securely on the cloud. I just re-installed the network drives this morning. To preserve data integrity, you must understand your data sources and how to use them.
Choosing Data Clouds
You can setup SQL database through well known and trusted clouds like Microsoft SQL, IBM Db2 Big SQL and Google Cloud SQL but there are many others out there who can provide online and offline iterations of your SQL database.
While you can setup MongoDB databases through a bigger bunch of much more providers you can find on the web, the MongoDB Atlas is the best way to get started as it has a free tier you can use for learning.
Each of these clouds have infrastructure requirements so it is important to learn about these properly before choosing.
Query and Referencing
When you want to start storing data from external sources, you would need a Database Management System, also known as DBMS. With this, you can quickly start exploring data as long as you organize your imports properly.
You can use an SQL database if you want to carry out analytics on the data or a document database like MongoDB if you would only make references to the database without analytical queries. Here are some examples:
- If you want to know how many cities a phone carrier has coverage in, you would need SQL.
- If you simply want to check the right phone carrier for a mobile number to choose SMS providers, you can use MongoDB.
Query and referencing are the major activities you would need to carry out first on your data before anyone else so you don't distribute inconsistent reports due to lack of structure. Let your scenario decide the type of DBMS you work with.
Single Source of Truth
It is important to choose a data cloud based on your needs not your comfort. While you start with free tiers, prepare for premium options so your business would not be impacted by feature limitations.
With your data cloud setup, you can start research. Just make sure your data is structured. That way you can enjoy the benefits of Data Integrity by delivering a single source of truth for your integrated business applications.
Without taking steps to structure internal data sources, it would be hard to obtain facts from research.
