The headless information structure is an natural emergence of the separation of knowledge storage, administration, optimization, and entry from the providers that write, course of, and question it. With this structure, you may handle your information from a single logical location, together with permissions, schema evolution, and desk optimizations. And, to high it off, it makes regulatory compliance so much easier, as a result of your information resides in a single place, as a substitute of being copied round to each processing engine that wants it.
We name it a “headless” information structure due to its similarity to a “headless server,” the place it’s a must to use your individual monitor and keyboard to log in. If you wish to course of or question your information in a headless information structure, you’ll have to deliver your individual processing or querying “head” and plug it into the information — for instance, Trino, Presto, Apache Flink, or Apache Spark.
A headless information structure can embody a number of information codecs, with information streams and tables as the 2 most typical. Streams present low-latency entry to incremental information, whereas tables present environment friendly bulk-query capabilities. Collectively, they provide the flexibility to decide on the format that’s most fitted on your use circumstances, whether or not it’s operational, analytical, or someplace in between.
First, let’s check out streaming within the headless information structure.
