Tutorial
Running Machine Learning Pipelines with Kedro, Kubeflow and Airflow
One of the biggest challenges of today’s Machine Learning world is the lack of standardization when it comes to models training. We all know that data…
Read moreUse-cases/Project
2 min read
Running Spark on Amazon Web Services (AWS)
When you search thought the net looking for methods of running Apache Spark on AWS infrastructure you are most likely to be redirected to the documentation of AWS EMR (Elastic Map Reduce) service, which is Amazon's Hadoop distribution suited to run in AWS cloud environment. It's quite an easy way to deploy your data pipelines, but sometimes bootstrapping a huge cluster to perform simple ad-hoc analysis it's a cumbersome task. They say:
"to a man with a hammer everything looks like a nail" :)
and we felt into this trap with EMR once.
The article below describes two other ways of running Apache Spark jobs on AWS-managed infrastructure - AWS Glue and AWS Fargate - that we use on our clients' data warehousing projects. You will find there the key differences between these methods when it comes to flexibility and pricing, showing why there is no place for "one service fits all" approach in AWS world.
Check out!
18 December 2019
Want more? Check our articles
Use-cases/Project
Puzzles in the time of plague: truly over-engineered audio spectrum analyzer
Quarantaine project Staying at home is not my particular strong point. But tough times have arrived and everybody needs to change their habits and re…
Read moreLarge Language Models - the legal aspects of licensing for commercial purposes
In the rapidly evolving landscape of artificial intelligence (AI), large language models (LLMs) have become indispensable tools for various…
Read moreWhitepaper
White Paper: Monitoring and Observability for Data Platform
About In this White Paper, we described a monitoring and observing data platform in case of continuously working processes. What you will find there…
Read moreUse-cases/Project
Real-time Machine Learning: considerations based on Fraud Detection use case
When it comes to machine learning, most products are designed to work in batches, meaning they process data at fixed intervals rather than in real…
Read moreTutorial
Stream enrichment with Flink SQL
In today's world, real-time data processing is essential for businesses that want to remain competitive and responsive. The ability to obtain results…
Read moreContact us
Interested in our solutions?
Contact us!
Contact us!
Together, we will select the best Big Data solutions for your organization and build a project that will have a real impact on your organization.
What did you find most impressive about GetInData?
They did a very good job in finding people that fitted in Acast both technically as well as culturally.
Type the form or send a e-mail: hello@getindata.com