Enabling Hive on Spark on CDH 5.14 — a few problems (and solutions)
Recently I’ve had an opportunity to configure CDH 5.14 Hadoop cluster of one of GetInData’s customers to make it possible to use Hive on Spark…
Read moreA partnership between iZettle and GetInData originated in the form of a two-day workshop focused on analyzing iZettle’s needs and exploring multiple cloud providers’ offerings. Outcomes of this event led to a year-long collaboration on building a robust, third wave data platform.
To ensure undisrupted business growth iZettle was looking for a data platform solution that could meet advanced analytics requirements and address the performance issues caused by rapidly swelling data collection. The platform should minimize the effort spent on maintenance work, allowing specialists to dedicate more time to exploratory data analysis and manufacturing business meaningful insights.
Daily loading jobs were replaced with a streaming ingestion process running on Google DataFlow. Currently, BigQuery takes the role of a central data lake and a query engine. The ingestion process uses an internal message dictionary to validate and route messages to relevant tables. Analytics work is orchestrated with Cloud Composer and utilises BigQuery for SQL and DataFlow for complex scenarios.
Recently I’ve had an opportunity to configure CDH 5.14 Hadoop cluster of one of GetInData’s customers to make it possible to use Hive on Spark…
Read moreIn a recent post on out Big Data blog, "Big Data for E-commerce", I wrote about how Big Data solutions are becoming indispensable in modern business…
Read moreApache Sedona is a distributed system which gives you the possibility to load, process, transform and analyze huge amounts of geospatial data across…
Read moreNowadays, we can see that AI/ML is visible everywhere, including advertising, healthcare, education, finance, automotive, public transport…
Read moreBig Data Technology Warsaw Summit 2020 is fast approaching. This will be 6th edition of the conference that is jointly organised by Evention and…
Read moreQuarantaine project Staying at home is not my particular strong point. But tough times have arrived and everybody needs to change their habits and re…
Read moreTogether, we will select the best Big Data solutions for your organization and build a project that will have a real impact on your organization.