Since the first day of planning, there was no doubt in any of our minds that we would be rebuilding the site from the ground up on a completely new set of technologies intended to do something quite different to our old site. The old site and CMS was a patch-work of code written by various developers at MWeb over the past 5 years, and the database was similarly messy, so there was no intention of trying to make it work with the new site.

The project scope and order of production was roughly this:

1. Redesign the front end
2. Design a new strategy for ad formats
3. Design a new CMS
4. Develop a new data structure that will be suitable for the very dynamic aspects of the site like real-time visualisation and lots of semantic tagging
5. Develop the CMS based on the editorial team’s current and future needs
6. Develop the site front-end
7. Migrate the existing data and archive into the new database
8. Contextually integrate the advertising engine
9. Negotiate with our advertisers to support the new advertising formats
10. Clean up the archive data manually
11. Hydrate the archive data with semantic data using Calais and some internal metadata
12. Integrate national-level mapping for navigation
13. Integrate city-level mapping using Google Maps
14. Integrate contextually-relevant external news sources using Google News
15. Integrate contextually relevant blog posts using Google blog search
16. Create new e-commerce partner site templates
17. Rebuild the newsletter system
18. Create smooth transitions between old data feed modules that were done in ASP.NET and the new ones
19. Integrate the swarm for real-time visualisation
20. Create the CMS dashboard showing real-time graphing of interactivity on the site
21. Migrate the old redirect system to a new system that does the same thing so the newspaper can mention short urls
22. Stress test the application server
23. Stress test the database
24. Soft launch the site to readers going to our old arts server and to those users who decide to click on the massive link we put on the top of the site
25. Optimise the database queries based on actual load
26. Optimise the caching strategy based on a blend of editorial currency and server load
27. Redirect the old site to the new site, monitor performance and continue optimisation under heavy load.

Aside from the web site UI design and project planning, which took 3 months of revisiting, the rest of the items on this list were done between 17 May and 22 June. Obviously because of the tight timeframe there were a lot of lessons learned on the fly and, to some extent, the story of this redevelopment is a great success story for the open-source community but also for close relationships between our business, editorial and technical teams.

The process we underwent to plan what we wanted and how it would be implemented was done primarily using the site look and feel process. I designed the front template, section template and article template based on a series of meetings with Matthew Buckland, the GM. In these meetings the primary concern was integration of news and Web 2 features into a news structure. Once the first set of designs were ready, Riaan Wolmarans, the editor joined the team and worked with me to get the layout and structure of the content right.

At the end of the process, the three templates were designed precisely as they would look on the screen, down to the line spacing in blurbs and the placement of the widgets, using real content. These templates acted as a blueprint when we started site construction. Take note, not a single line of text was written into a technical spec, the front-end was the spec and the CMS and site were built backwards from there.

Now that we’re over usual paranoias about going live, I’ll have some time to discuss some of the more interesting decisions/experiences, so keep coming back.