Why glomex is running on AWS cloud

· by Johannes Brandstetter · Read in about 5 min · (1030 words)

To be honest, using anything but Amazon Web Services (AWS) cloud at glomex was never even subject to discussion. glomex engineering and management completely trust AWS to provide us with secure, scalable and innovative services. On conferences such as data2day in Karlsruhe or CeBIT in Hannover, engineers and managers asked me why we were using AWS and how we convinced our management to use it. In this blog post, I’d like to describe our main reasons for using AWS and how we solved existing concerns.

Scaling

For the glomex business vision, a scalable IT infrastructure and application is very important. We never know how many publishers will integrate our video player into their webpages and how many end users will watch videos via our video players. We can see peaks growing by factor 5 up to 50 within minutes:

scaling

All our infrastructure and applications are designed to manage these kinds of peaks. We use Amazon AutoScaling, our Amazon RDS databases use read-replicas and, for the click-stream dataflow, we use Amazon Kinesis Firehose which scales automatically. In all infrastructure areas, AWS provides us with a working solution for scaling infrastructure or at least guidelines to make them scale.

Global Footprint

Furthermore, our business vision aims at content owners and publishers around the world. From day one, we were interested in building a system which we can scale globally. AWS provides us with 13 regions (as of October 2016) around the world. We do not have to negotiate prices or deals with infrastructure providers around the world. With AWS, we have one bill and identical computer centers around the world accessible via one API.

Automation

Access to infrastructure around the world sounds great. But how to manage this? AWS provides us tools to easily deploy our applications in all regions with the same process. We have all our infrastructure defined via Amazon CloudFront (Infrastructure as a code) and use tools such as Amazon CodeDeploy or Amazon ElasticBeanstalk to ship our application. To fulfill our requirements, we developed some additional tooling around this to make it more user friendly and more standardized for all our teams. Having everything fully automated, it takes us minutes to bring up our complete infrastructure and applications wherever we have new customers.

High Availability & Disaster Recovery

Videos have to be constantly available and especially available around special social or political events. The AWS multi-availability-zone concept automatically provides us with a highly available infrastructure setup out of the box. In detail, our AutoScaling groups start servers on load request and distribute these servers automatically about two or three availability zones, which are separated computer centers. Having all this automated in code, we do not need a disaster recovery computer center running on hot standby. In case of an AWS region blackout (never happened in the last ten years), we are able to bring glomex up and running in another AWS region within a few hours. We plan to introduce a quarterly “disaster game day” where we test exactly that scenario.

Costs & Managed Services

Reducing cost was another reason for choosing the cloud. We only run and pay servers when they are required. E.g. we switch off development infrastructure over the weekend. In our AutoScaling groups and Amazon EMR Hadoop clusters, we use spot instances and we choose AWS managed services whenever possible. In this case, we can reduce the operations team and focus much more on our real business vision and development of the glomex application (instead of infrastructure management).

Agile & AWS Breath & Time to Market

With this new focus, our engineers can be much more creative and design better applications. AWS supports our agile development processes with their platform breath. If our engineers want to switch from PostgreSQL to MySQL, they can do so within minutes via Amazon RDS, and if a new caching layer is required, we can start Amazon ElasticCache within minutes without ordering and buying new hardware, taking months. By completely automating the infrastructure development and operations process and putting it into the normal software development lifecycle, we are able to focus on providing business value to our customers in the fastest way possible.

Security

Last but not least, security is one of our biggest concerns. We talked to the ProSiebenSat.1 security officer about how to build a secure system in the cloud and how to fulfill all legal requirements several times. AWS provides lots of useful documentation around security and complies to many certifications like HIPPA or PCI DSS. Those are certifications from independent agencies, setting high standards that have to be fulfilled by AWS. We trust AWS to run their services in a secure environment. At the end we have to guarantee the security of the system we are running on AWS (compare shared responsibility model). Main pillars of our security concept are:

  • Everything runs in our own VPC
  • VPCs are connected via a VPN with our network
  • AWS Security groups are actively monitored
  • Netflix Security Monkey is actively checking for security issues
  • We use MFA access for AWS and github wherever possible
  • All European customer data stays in Europe (Ireland)
  • As a first processing step, we anonymize all our data
  • Business-critical data is encrypted via Amazon KMS

Summary

We never had any concerns that AWS cloud wouldn’t fit our requirements. AWS provides us a scaling, global and highly available infrastructure. We can easily develop a fully automated infrastructure for low cost and with low operations duty. Security requirements are completely fulfilled. We are aware that we are responsible for the security of our application and have several tools in place to monitor and track our security. We are also aware that we have a strong vendor-lock-in and that AWS is listening to our feature requests and already released one feature and two bug fixes related to our request.

For more check the video of our CTO Michael at the AWS Summit Berlin:

Find out how glomex can help you monetize video content at info@glomex.com or join our creative and agile development team to focus on glomex business value running on AWS.

by Markus Schmidberger, Architect Data Service Johannes Brandstetter, Head of Operations Michael Muckel, Vice President of Engineering