In this series, I will be taking real-world services and processes, and modelling them using either GCP or AWS. The idea is to give a behind the scenes look at how these products would operate on a cloud platform. For reference, I am both GCP and AWS certified.
Why F1 Analytics?
Motorsport is man and machine pushing themselves to the extreme. Nowhere is this truer than Formula 1 (F1), it is one of the most hardcore and technically advanced race series out there. The difference between winning and losing can be the difference of a 1000th of a second. As a result, F1 teams go to extreme lengths using the latest equipment in the hope it gives them the edge.
Each car is the result of countless hours of processing and analysing data. This data comes from a variety of sources. At the factory, Wind Tunnels and CFD applications are used to model new parts. During a race, sensors on the car stream data which must be processed in realtime to give the teams the best opportunity to make a potentially race-winning decision.
Up to 45TB of data a week, [1] can be generated through these processes. Having the best technology at your disposal to generate the insights and information you need is vital.
Design
Why these Services?
Pub/Sub
Effortlessly scaling to receive millions of messages a second, makes it perfect to receive real-time data from IoT sensors on the car.
Dataflow
Handles both Batch and Stream inputs to perfectly cater for all ETL scenarios.
Kubernetes Engine
Containers use auto-scaling to handle big processing jobs with minimal setup. They are the perfect tool to balance powerful capabilities and cost/efficiency constraints.
BigQuery
Limitless capacity and ultra-fast processing of up to petabytes worth of data creates the perfect tool to analyse and store the data generated.
Bigtable
Highly resilient storage offering with limitless scaling and low latency, making it the tool of choice for storing IoT and Wind Tunnel streams.
Cloud Storage
Perfect for storing output files from analysing data and having them accessible at both the track and factory.
Datalab
Visualise and explore BigQuery results for easier understanding and sharing.
Machine Learning
Utilise the vast data on offer to create better predictive models and process data to gain additional insights.
Dedicated Interconnect
A high bandwidth, resilient connection direct into Google’s infrastructure provides better performance for performing large workloads.
Direct Peering
Low latency connection into Google’s infrastructure from 100+ locations, makes it perfect for visiting racetracks across the world.
IAM
Use IAM to secure access to the data and make sure only the correct people are authorised to perform certain tasks.
Stackdriver
An all in one monitoring and logging tool, designed to provide visibility of everything happening within the environment and alert when something unexpected happens.
