Data Engineering
A Comparative Study of Different Web Crawler Frameworks

What is a web crawler? A website crawler (or web crawler) is a piece of software that does the process of hunting down information…
Data Engineering
SPARK SHUFFLE TUNING

WHEN DOES SHUFFLE OCCUR? A shuffle can occur when the resulting RDD from a transformation depends on other elements from the same or another…
Data Engineering
SEMANTIC GRAPHS

In an enterprise, there are several challenges to data management among multiple people, departments, and processes. The current big data revolution is generating a…