Continuous Deployment and Delivery Server for Code, Content, and Data
2K is a California-based video game publisher founded in 2005 within the Take Two Interactive holding, which is one of the largest publishers in the U.S. With ever more critically-acclaimed titles to their name, 2K came to the point where having to release multiple marketing websites over short periods of time had become increasingly challenging. In order to address the company’s increasing need for rapid production of numerous sites, 2K’s marketing team faced a choice between scaling up their team, or finding a smarter way to work that would also allow for a better workflow across the organization. Opting for the latter, they partnered with Xolvio, who responded to their challenges by ideating the Futurama project, an ecosystem of solutions designed to transform the way 2K create and release websites. The first of these solutions was a bespoke digital platform: Continuous Deployment and Delivery Server for Code, Content and Data aka CD3, whose aim was to dramatically reduce infrastructure costs and automate the release process. As a result, CD3 allowed for a massive scaling up of 2K’s capabilities to release websites in terms of both speed and volume, saving the company around $1,000,000 per year.
With a growing number of titles planned for release, 2K’s development team found themselves under immense pressure to keep up the tempo. The need for automation, scalability and speed encompassed several areas of the company’s web development processes:
The amount of work needed from 2K’s infrastructure & operations team posed a significant problem due to their limited time and resources. Freeing them up from the burden of performing manual updates to cloud assets, building servers, and whatnots was a crucial goal.
The marketing and legal teams always need to preview websites before they are deployed to production. The development team catered to this demand by creating different environments on a per site basis, raising the need for automation, so that their time could be better used elsewhere.
Websites are ever-changing and 2K is no exception. Sometimes things don’t go according to plan or approval is withdrawn for whatever reason, posing the need to be able to quickly roll back to a previous version.
Every time the marketing team created changes, the development team had to release them to production. In order to reduce errors and gaps in communication, 2K needed to extend release capability beyond the development team by making the release process self-service for all.
Given the scope of infrastructure and release challenges 2K faced, we employed our proprietary Quality Faster process to the project. The work started with the Discover phase, where we learned about our client’s challenges and problems in detail, applying tried and tested techniques like Event Storming, interviews and discovery sessions to arrive at a shared understanding of what needs to be achieved. This led to formulating the concept of the Continuous Deployment and Delivery Server for Code, Content and Data, or CD3 for short. The solution's main goal was to remove all manual processes associated with infrastructure environments and release, in order to achieve maximum speed and scalability.
Over the first 4 weeks of our engagement, we went from establishing the requirements and the problem domain to carefully carving out the solution domain. As we progressed with the design through deeper collaborative architecture sessions, the CD3 concept quickly matured into a living design and an actionable delivery plan. By means of our proprietary technique called the State of Integration Diagram, we divided up the work to be done and planned our iterations over the course of 4 more weeks. By the end of week 8 of the project, the initial version of CD3 was fully operational and was employed to release 2K’s highest profile title, NBA2K22.
The CD3 solution automated all of the laborious tasks that were previously burdening 2K’s infrastructure & operations team. To achieve this, we leveraged Amazon’s CDK toolset to create and manage cloud resources through infrastructure as code practices. As part of this approach, we used Amazon’s content delivery network, CloudFront, and built several capabilities and integrations on top of it, remoulding the CDN into a highly customized web server. At its core, CD3 is provided with a build containing code, content and data, which is then taken through an environment promotion pipeline. Let’s see how this works in more detail:
Whenever a developer pushes code to GitHub, an action is triggered which takes the code through a rigorous automated testing cycle, before being pre-rendered and forming a new build in the build repository. The repository treats code, content, and data as independently releasable modules that can be deployed to any environment.
2K’s content authors use 2 CMS solutions, Cockpit and Contentful, which were both integrated into CD3 until the future full migration to Contentful. Adding new content triggers a webhook that pushes the content to be pre-rendered and added to the build repository, just like code. An automated workflow based on GitHub actions deploys the new content to various environments.
Several data sources were integrated into CD3 such as social media feeds, pricing data, Google sheets, and more. By leveraging architectural patterns such as repositories, we were able to separate the data concerns from content to code, making it easier and therefore quicker to build and extend data import processes.
One of the project’s important goals was to automate the creation of preview environments for approval and legal reviews. These environments allow the review of any combination of code content and data by the respective teams.
CD3’s release management center, the Admin UI is used to promote specific builds to production or to ad hoc environments, e.g. for security review. The Admin UI allows the development team to select any build version (code, content and data) and release to an environment within seconds. Rollback to any previous version is also possible and equally fast.
In order to make sure every page loads within milliseconds for the best user experience, we made heavy use of pre-rendering and caching techniques. We created custom tooling that allowed the 2K development team to code at breakneck speeds, and for the builds they produce to be automatically pre-rendered for every single page, for every language, and for every region. These builds are then uploaded to S3, Amazon’s storage service, and served through CloudFront at blistering speeds.
Through ideating and developing the CD3 solution, we directly responded to all of 2K’s site development challenges, as well as improved additional aspects along the way. The outcomes of the project include:
No more laborious manual tasks required on the part of the infrastructure & operations team thanks to automating infrastructure as code.
Instead of wasting time on setting up environments e.g. for legal review, everyone can now see what they need to see, instantly.
Feedback withdrawal or any content errors have become non-issues thanks to the ability to roll back to a previous version within seconds. Further, the decoupling of code, content, and data allows for immense flexibility in release management.
Any website changes can now be released live with next to no effort, so that each and every team can focus on their respective work areas.
By including pre-rendering automation within the solution, site load times have become lightning fast by default, which has a direct impact on e-commerce conversion.
All of the above benefits jointly contributed to a substantial increase in speed to market for 2K’s marketing team, making it much easier for them to follow the company’s tight release schedule. At the same time, all of CD3’s automation reduced the need for extra headcount. Overall, the resulting optimization in terms of infrastructure costs, labor costs, and speed allowed 2K to save around $1,000,000 a year.
Do yourself a favor and reach out to solve your digital problems
Pick our brains